WO1999017265A1

WO1999017265A1 - Method and apparatus for surgical training and simulating surgery

Info

Publication number: WO1999017265A1
Application number: PCT/US1998/019964
Authority: WO
Inventors: Robert Playter; Marc Raibert
Original assignee: Boston Dynamics, Inc.
Priority date: 1997-09-26
Filing date: 1998-09-25
Publication date: 1999-04-08

Abstract

A real-time surgical simulation system includes a body environment simulator and a surgical tools simulator. The body environment simulator simulates a body environment having components including, for example, bones, joints, tissues, organs and the like. The surgical tools simulator simulates at least one surgical tool that can interact with the simulated body environment, and that is coupled to a surgical tool that is manipulated by a user of the surgical simulation system of the invention. The surgical simulation system also includes a visual display device for displaying the simulated body environment and the simulated surgical tool. The surgical simulation system further includes an interaction simulator and a force presenting module. The interaction simulator means simulates, in real-time, interaction between any of the simulated body environment, components of the simulated body environment and the simulated surgical tool. This simulated interaction is provided on the visual display device to the user. The force presenting module presents simulated forces of the simulated interaction at the surgical tool to the user of the simulated surgical system. With this system, there is provided an apparatus and a process for learning, practicing and experiencing dynamic, real-time, surgical procedures having a life-like feel, graphics and sound of surgical procedures.

Description

METHOD AND APPARATUS FOR SURGICAL TRAINING AND SIMULATING SURGERY

This Application claims the benefit under 35 U.S.C. §119(e) to Provisional Application No. 60/060,108 filed September 26. 1997

Background of the Invention

1. Field of the Invention

The invention relates generally to a computer-based simulation system of surgical procedures. In particular, the invention relates to a system for providing a realistic simulation of surgical procedures such as anastomosis.

2. Discussion of the Related Art

Modern medicine has given rise to an increasing number of highly sophisticated surgical procedures that are performed on patients for a variety of purposes. For many of these surgical procedures, it has been the case that the only way for a practitioner to master the necessary skills and required techniques was through experience on live subjects, or animals. Live patients or animals have been essential in practicing these surgical procedures because the normal pressures, touch, sound, and relevant patient response are important, if not crucial elements of practicing successful surgical procedures. Accordingly, the tendency has been for already experienced physicians to be asked to perform whatever procedures are necessary, and it has been difficult for inexperienced physicians to obtain a desired level of competence. In addition, there is a need to maintain a high degree of skill for even experienced physicians, that is only possible through continued training. Accordingly, there is a continuing need for a substitute way of providing life-like experience with out the necessity for obtaining the experience on living persons or animals.

Proposals have been made to simulate living conditions in non-living substitutes. For example, the use of computer-assisted simulation technology is becoming applicable in medicine, in part because of the high cost of traditional training resources such as live animals, physicians' time, and the like. Accordingly, the value of computer-assisted instruction has been recognized. Indeed, during the past decade, numerous medical centers have designed and used software for medical education. The software has been primarily designed to run on, for example, an IBM PC compatible and/or Macintosh computers, may include interactive capabilities with the user and allows for self-paced instruction of the user. Many of the programs also have simulations which require the user to make patient management decisions. More recently, medical education programs have used computer graphics for instruction of a patient's anatomy. These programs provide the advantage of giving a student an opportunity to access anatomic images and related textural information, rather than spending numerous hours dissecting cadavers in a lab, and provide the additional benefit of allowing the student to spend as much time as needed to learn the material. Although these computer-assisted programs offer many advantages in medical education, there are many limitations and drawbacks such as, for example, there is a limited scope of depicted anatomical information, and these programs do not allow for practice of actual medical procedures under realistic conditions.

One proposal that has been made to simulate living conditions in non-living substitutes is U.S. Patent No. 4,907,973, granted on March 13, 1990 to David C. Hon

(hereinafter the Hon patent). Referring to Fig. 1, the Hon patent discloses a computer-based surgical simulation system 100 including a peripheral device designed to simulate a surgical instrument, as well as the involved anatomy of a patient. In particular, the Hon patent discloses in one embodiment a system for computer simulation of an endoscope examination. The Hon system includes a mock catheter 102 that is inserted within a model 104 or mock arterial path of a physical internal body part of a model. Within the model, there are a plurality of sensors (not illustrated) on the mock arterial path that track the progress of the catheter through the mock arterial path. The sensors transmit corresponding signals 105 to a computer 106 which responds to the signals by accessing stored video information in storage medium 108 representing the view which would be observed from the relative location of a tip of the catheter device. The video representation 107 is then presented by a video display 108 to the user 110 of the system of the Hon patent. In one embodiment of the system of the Hon patent, a vessel constricting simulator within the mock arterial path provides a fixed resistance to the progress of the catheter. While the system of the Hon patent has the potential for providing a feel of a catheter device, it suffers a number of infirmities. For example, the system of the Hon patent lacks flexibility for easily altering the involved anatomy and procedures; specifically it lacks the flexibility for providing different anatomies, patient conditions and procedures which would require additional physical models for each as well as additional stored video information for each. In addition, the system of the Hon patent lacks the flexibility to realistically simulate changes in the mock environment responsive to manipulations and performance of the surgical instrument by the operator of the system. By way of example, the system of the Hon patent does not provide for the realistic simulation of bleeding, e.g., in the event an arterial wall is punctured by the catheter, or catheter balloon inflation and consequent removal of an occlusion within the mock arterial path. In addition, the system of the Hon patent does not use dynamic simulation of the body environment. Instead, the displayed indicia representing the view from the catheter is a prestored video sequence.

Moreover, the system of the Hon patent is oversimplified and less than realistic. For example, the system of the Hon patent does not provide for tracking and displaying rotation of the catheter device in the mock body environment; can only apply passive breaking forces to the catheter by the constrictors along the mock arterial path and therefore cannot simulate active forces such as those from a beating heart or a pulsing artery; the forces applied cannot be forces that result from movement of the surgical tool within the mock body environment and thus movement of the tool within the body environment cannot be simulated; the image representing the internal body environment cannot be modified in real-time as a result of any interaction of the catheter within the mock body environment and therefore simulation of interaction such as, for example, deformation of tissue through interaction with the catheter is not possible; the system is limited to camera based instruments like endoscopes and catheters which travel inside tubes inside the body; and the system does not have the ability to simulate interaction of the mock body environment with the surgical tools by changing the mock body environment shape, appearance, or position in response to any of poking, plucking, puncturing or incising of the mock body environment.

The Hon patent also discloses that surgical performances of the user can be recorded for evaluation, review, and comparison to other performances. However, because the system of the Hon patent suffers the infirmities discussed above, in particular, does not provide for contact or interaction between the mock body environment and the mock surgical tools, the system of the Hon patent cannot record, compare, evaluate and score performances of a user based on forces of interaction between the body environment and the surgical tools. Therefore, the system of the Hon patent is over-simplified and less than realistic.

Another proposal for a system for providing a realistic simulation of a cardiac catheterization procedure is disclosed in international publication No. WO 96/28800, published on September 19, 1996 (the Merril disclosure). Referring to Fig. 2, the medical simulation system of the Merril disclosure includes a computer-human interface device 112 and a computer 114 storing a software program which works in conjunction with the computer-human interface device. In a preferred embodiment of the Merril disclosure, the system simulates a cardiac catheterization procedure, and the interface device is a catheter interface device that tracks the translational and rotational movement of a mock catheter 116, and provides a signal 118 to the computer program. The computer program generates a tactile feedback force signal 120 that is output to the catheter interface device to simulate resistance to the mock catheter. The computer program also provides a virtual reality simulation of movement of the catheter within the virtual catheter on a high-resolution monitor 122. The computer program can provide a variety of enhancements including 3-D imaging, narrative audio or music via a CD-ROM 124, simulated radiopaque dye infusion, bleeding and atrial pulsation in synchronism with an on-screen EKG display.

While the Merril disclosure provides a more realistic simulation than that of the Hon system, it also suffer a number of infirmities. In particular, the system of the Merril disclosure does not provide for simulated objects and tools within a simulated body environment that have their own position, orientation, space, and that can move independently in the simulated body environment. The system of the Merril disclosure does not provide for contact between any of the simulated objects and tools within the simulated body environment. Moreover, the system of the Merril disclosure does not allow for the catheter to have a position independent of the simulated body environment that can be moved independent of the body environment. The system of the Merril disclosure is also limited to '"canned" or pre-stored simulations of simulated vessel movement, simulated catheter positioning and simulated vessel expansion in response to simulated angioplasty balloon inflation. These are precomputed responses that are not responsive to any interaction between the catheter and the simulated arterial tree. In addition, the system of the Merril disclosure can only provide passive breaking forces to the mock catheter as discussed above with respect to the Hon patent, and cannot simulate active forces such as from a beating heart or a pulsing artery.

Accordingly, the system of the Merril disclosure is also less than realistic.

Summary of the Invention

In view of the foregoing, it is an object of the invention to provide an improved system for simulating surgical procedures and for training. According to one embodiment of the invention, a real-time surgical simulation system includes a body environment simulator and a surgical tools simulator. The body environment simulator simulates a body environment having components including any of, for example, bones, joints, tissues, organs and the like. The surgical tools simulator simulates at least one surgical tool that can interact with the simulated body environment, and that is coupled to a surgical tool that is manipulated by a user of the surgical simulation system of the invention.

The surgical simulation system also includes a visual display device for displaying the simulated body environment and the at least one simulated surgical tool. The surgical simulation system further includes an interaction simulator and a force presenting device. The interaction simulator simulates, in real-time, interaction between any of the simulated body environment, components of the simulated body environment and the at least one simulated surgical tool. The interaction simulator also simulates interaction between simulated surgical tools. The simulated interaction is provided on the visual display device to the user. The force presenting device simulates and presents forces of the simulated interaction at the surgical tool manipulated by the user of the simulated surgical system. With this system, a realistic simulated surgical procedure, including dynamic real-time, life-like sensation, graphics and sound is provided to the user to learn, practice and experience surgical procedures.

According to another embodiment of the real-time surgical simulation system of the invention, the surgical simulation system includes a collision detector that detects collisions between the simulated surgical tool and either of the simulated body environment, the components of the simulated body environment and the other simulated surgical tools. The interaction simulator of this embodiment computes the exchange of forces between the simulated surgical tool and any of the simulated body environment, the components of the simulated body environment and the other simulated surgical tools and determines movements and deformation of any of the simulated surgical tool, the body environment and the components of the body environment..

According to another embodiment, the surgical simulation system includes a storage media that can be used to record any of surgical tool position and orientation, simulated body environment and simulated body component position and orientation, their interaction forces, and an elapsed time of the simulated surgical procedure. This embodiment further includes a surgical skill scoring module that analyzes the above-mentioned recorded data to produce a measure of surgical skill, a comparator that compares recorded data of the user performing the simulated surgical procedure with previously recorded data of, for example, an "expert" who has performed the same simulated surgical procedure, and an analyzer that analyzes an output of the comparator to provide an assessment of the performance of the user of the system. With this embodiment, the surgical simulation system makes it possible to objectively evaluate the surgical skills of a user of the system. In particular, the user of the system will be able to review their performance and compare it with that of "experts". Moreover, the system provides a standard to the user, and enhances the teaching capacity of "experts" in the field beyond a one-to-one apprenticeship. Moreover, the system can be used to accredit the user for a particular surgical procedure.

In still another embodiment of the surgical simulation system of the invention, the body environment simulator includes a plurality of models of tube-like organs, each having any of, for example, a length, width, thickness, mass, inertia, compliance, puncture resistance, friction, surface texture, and surface color information. The surgical tools simulator simulates forceps, a needle holder, a needle and a thread. With this embodiment, a surgical simulation of anastomosis of tube-like organs, wherein the tube-like organs can be sewn together by the user with the simulated needle and thread, can be performed by the user.

A method of providing an interactive, real-time surgical simulation according to the invention includes simulating a body environment having components including any of, for example, bones, joints, tissues, and organs, and simulating at least one surgical tool that interacts with the simulated body environment. According to the method, the simulated surgical tool is controllable by a user of the method with a physical tool that is manipulated by the user of the method. The method includes the step of displaying the simulated body environment and the simulated surgical tool to the user. The method also includes the steps of simulating, in real-time, interaction between any of the simulated body environment, components of the body environment and the simulated surgical tool and the step of presenting the simulated forces of the simulated interaction to the user of the method at the surgical tool. With this method, a process for learning, practicing and experiencing dynamic, real-time surgical procedures including life-like graphics, feel and sound is available to a user of the method. According to another embodiment of the method of the invention, the step of simulating the interaction includes detecting a collision between the simulated surgical tool and either of the simulated body environment and the components of the simulated body environment. The step of simulating the interaction also includes determining the exchange of force between the simulated surgical tool and either of the simulated body environment and the components of the simulated body environment and determining any movement and deformation of any of the simulated surgical tool, the body environment and the components of the body environment. According to still another embodiment, the method includes the step of recording data representing any of, for example, surgical tool position and orientation, simulated body environment and components of the body environment position and orientation, interaction forces, and an elapsed time of the simulated medical procedure. This embodiment of the method further includes the step of comparing the recorded data with recorded data of. for example, an '"expert" who has also performed the method of the invention, and evaluating the comparison data to provide an assessment of the performance by the user of the method of the invention. With this embodiment of the method of the invention, it is possible to objectively evaluate the surgical skills of the user. In addition, it is possible for the user to review his or her performance and compare it to that of "experts" who have performed the same procedure and by using the recorded performance of the "expert" as a standard, the method of the invention enhances the teaching capacity of the '"expert" beyond a one-to-one relationship. Moreover, the method can be used as a process for accrediting the user for a particular surgical procedure.

In another embodiment of the method of the invention, the step of simulating the body environment includes simulating a plurality of tube-like organs having any of, for example, length, width, thickness, mass, inertia, compliance, puncture resistance, friction, surface texture, and surface color information. In this embodiment, the step of simulating at least one surgical tool includes simulating forceps, a needle holder, a needle and thread, and the step of simulating the interaction includes simulating the surgical procedure of anastomosis of the tube-like organs, wherein the tube-like organs are sewn together with the needle and thread. Brief Description of the Drawings

Other objects and features of the present invention will become apparent from the following detailed description when taken in connection with the following drawings. It is to be understood that the drawings are for the purpose of illustration only and are not intended as a definition of the limits of the invention. The foregoing and other objects and advantages of the invention will become more clear with reference to the following detailed description of the drawings, in which like elements have been given like reference characters, and in which:

Fig. 1 illustrates a computer-based surgical simulation system according to the prior art;

Fig. 2 illustrates another embodiment of a surgical simulation system according to the prior art:

Fig. 3 illustrates a preferred embodiment of a surgical simulation system of the invention;

Fig. 4 is a perspective view of a housing of the surgical simulation system of Fig. 3. and a user working at the housing; Fig. 5 illustrates a top view of the user's perspective of a simulated surgical procedure provided by the surgical simulation system of Fig. 3;

Fig. 6 is a flow diagram of a surgical skill evaluation method and module of the surgical simulation system of Fig. 3;

Fig. 7 illustrates an example of a display screen provided by the surgical simulation system of the invention;

Fig. 8 illustrates an embodiment of a surgical skill evaluation module of the invention; Fig. 9 illustrates a method of measuring accuracy of needle placement by a user of the surgical simulation system of the invention;

Fig. 10 illustrates a method of measuring any surface damage to the simulated body environment by a user of the surgical simulation system of the invention;

Fig. 1 1 illustrates a method of measuring any tissue damage to the simulated body environment by a user of the surgical simulation system of the invention;

Fig. 12 illustrates a method of measuring a suturing technique by a user of the surgical simulation system of the invention; Fig. 13 is a block diagram of an embodiment of a processor of the surgical simulation system of the invention; Fig. 14 is a block diagram of an embodiment of a 3-D graphic simulator within the surgical simulation system of the invention;

Fig. 15A illustrates a cross-sectional view of deformation of a simulated body object according to a deformation module of the surgical simulation system of the invention;

Fig. 15B illustrates a perspective view of the deformed body object of Fig. 15A; Fig. 16 is a block diagram of an embodiment of a dynamic simulator within the surgical simulation system of the invention;

Fig. 17 illustrates a method of movement modeling by the surgical simulation system of the invention;

Fig. 18 is a block diagram of another embodiment of a dynamic simulator of anastomosis of tube-like organs according to an embodiment of the surgical simulation system of the invention;

Fig. 19 illustrates a tube model according to one embodiment of the surgical simulation system of the invention;

Fig. 20 illustrates another embodiment of a simple dynamic tube model according to the invention; and

Fig. 21 illustrates still another embodiment of a simple dynamic tube model according to the surgical simulation system of the invention.

Detailed Description It is to be understood that for this application, an active force-feedback device is a device that provides forces to surgical tools connected to the active force-feedback device. According to this invention, it is to be understood that anastomosis is defined to be the suturing of any tube-like organs such as, for example, blood vessels, the esophagus, ureters, bronchi, the colon, ducts and other organs found throughout the human body. A preferred embodiment of a surgical simulation system 10 of the invention is illustrated in Fig. 3. The surgical simulation system includes several elements that work in concert including: a simulation processor 12 such as, for example, a personal computer, a graphics processor 13 and a display 14 such as, for example, an Octane computer graphics workstation and monitor made by Silicon Graphics. Inc.; a computer communications network 15 connecting the two processors, such as, for example, an ethernet; a pair of Crystal Eyes glasses 48 and emitter which work in concert with the computer graphics display to create a three-dimensional (3-D) visual display of the computer graphics; force-feedback devices 16, 18, manufactured by, for example, SensAble Technologies of Cambridge, Massachusetts that provide active force feedback; surgical tools such as, for example, forceps 20 and needle holder 22 fit with sensors 21. 23 to measure tool configuration such as, for example, jaw angles; and a housing 24 that houses these components. It is to be appreciated that the simulation processor and computer graphics processor can exist in either multiple distinct computers as described above, in one computer with multiple processors or even in a single computer processor. It is also to be appreciated that although forceps and a needle holder are illustrated, any surgical tools are within the scope of the invention and can include, for example, those used in arthroscopic surgery. The active force feedback device 16, 18 coupled to the respective surgical tool 20. 22 applies forces having a magnitude and direction at a point of attachment of the force feedback device to the surgical tool. Each force-feedback device in coordination with the sensors fit to the surgical tools measures a position, orientation and configuration of the surgical tools. As will be described in greater detail below, the processor 12 of the surgical system of the invention simulates simulated surgical tools within a simulated body environment. The simulated surgical tools have a position, orientation and movement that corresponds to the surgical tools. The forces applied to the surgical tool by the force feedback devices are used to simulate forces of interaction between the simulated surgical tool and the simulated body environment simulated by the processor and displayed on the 3-D display. With this system. a user of the system holding and manipulating the surgical tools will feel forces through the surgical tools that simulate the forces the user would feel if the tool were being used in a surgical procedure.

As will also be discussed in greater detail below, the simulation processor 12 of the surgical simulation system of Fig. 3 includes a dynamic simulator 78 of a body environment including, for example, bones, joints, body tissues, organs, ligaments, cartilage, and muscle tissues. The dynamic simulator may be, for example, custom C and C++ Software running at 500-1000 Hz on the personal computer, that provides a realistic simulation of the body environment. The surgical simulation system detects contact between and simulates forces of interaction between the simulated surgical tools and between the simulated surgical tools and the simulated body environment or between elements of the simulated body environment, and simulates the movement of the body environment or elements of the body environment in response to those interaction forces.

The computer graphics processor 13 of the surgical simulation system of Fig. 3 includes a graphics simulator 74 of a body environment and surgical tools. The graphics simulator may be, for example, custom C and C++ and Open GL Software running at 20-30 Hz. The graphics display 14 portrays a computer graphic image of the simulated body environment and surgical tools. The simulated body environment and surgical tools portrayed by the surgical simulation system can be displayed in either 3-D or two-dimensional (2-D) formats and. as will be discussed in greater detail below, can be simulated from any viewpoint including the viewpoint of the surgeon's perspective or a viewpoint of, for example, an arthroscopic camera inserted into the simulated surgical environment. Moreover, photographic texture mapping and realistic deformable 3-D geometric models to be discussed in greater detail below, may also be used to make the images realistic and to show the effect of forces of interaction such as, for example, tissue deformation.

The surgical simulation system 10 of the invention simulates forces of interaction between the simulated body environment and the surgical tools 20, 22. The surgical simulation systems includes the active force feedback devices 16, 18 which provide interactive force-feedback with the surgical tools. The respective force feedback devices measure a position and orientation of the attachment of the surgical tools 20, 22 and provide that information to the processor. Furthermore, the sensors 21. 23 on the surgical tools measure the configuration of the tool such as, for example, jaw positions. As will be discussed in greater detail below, the simulation processor uses the position, orientation, and tool configuration information from the active force feedback device and surgical tool sensors to determine the position and orientation of the surgical tools in the simulated body environment. This information is combined with position and orientation information of the simulated body environment to detect contact between the surgical tools and the simulated body environment and to compute active forces to be applied to the force feedback devices representative of interaction between the surgical tools and the simulated body environment. This active force information is supplied to the force feedback devices 16, 18.

As will also be discussed in greater detail below, the force-feedback devices 16, 18 when combined with physics-based simulation by the dynamic simulator 78 of the body environment, provide realistic constraints on manipulation of the surgical tools within the simulated body environment. Accordingly, when the user of the system touches any component of the simulated body environment with the simulated surgical tools, the user will feel it by the forces presented to the surgical tools through the active force feedback devices. Simulated forces of interaction are also applied to the elements of the body environment which respond to the forces in realistic ways as provided by physics-based simulation of the simulated body environment.

Moreover, as will be described in greater detail below, the simulation processor 12 includes a collision detection module 86 that detects collisions between the user-controlled simulated surgical tools and the simulated components of the body environment or between two or more simulated components of the body environment. The simulation processor also includes a contact force modeling module 88 that models the contact force between the simulated surgical tools and the components of the simulated body environment or between two or more simulated components of the body environment and provides an output to a movement modeling module 90 that models and computes the response or movement of the simulated body environment to the forces resulting from this contact. The contact forces are also applied to the surgical tools through the active force feedback devices.

With the surgical simulation system of the invention, when the user of the surgical simulation system touches any component of the simulated body environment with the simulated surgical tool, the user will feel it at the surgical tool and the computed interaction forces will prevent the user from doing something that is not possible such as, for example. pushing the simulated surgical tool through a simulated organ without resistance. One problem with prior art surgical simulators that do not include any of the active force-feedback devices, collision modeling, contact force modeling and movement modeling is that a simulated object occupying a same physical space of another simulated object will pass completely through one another as though they are not there. Therefore, an advantage of the surgical simulation system of the invention, in contrast to a simulation system of the prior art is that the surgical simulation system of the invention provides the user tangible interaction with the simulated body environment in the form of active force-feedback and visible simulated body environment movement in response to interaction forces.

Fig. 4 is a perspective view of the housing 24 containing the surgical simulation system of Fig. 3 and a user 46 working at the housing. The user holds the surgical tools 20.

22 in his/her hands and looks through the 3-D glasses 48 at the simulated surgical tools and simulated body environment presented on the 3-D graphics display 14 (See Fig. 3). In a preferred embodiment of the housing and the surgical simulation system of the invention, a visual image of the simulated body environment is displayed with the 3-D graphics display and a haptic image that is accessible to the user^'s sense of touch with the dynamic simulator, the force feedback devices, and the surgical tools is also displayed. The visual images of the body environment are displayed through a mirror system 50 that has the effect of placing the visual work space in front of the user at approximately waist level of the user as is illustrated in Fig. 4. This arrangement allows the visual images of the simulated body environment to be nearly co-located in space with the haptic image of the components of the body environment and the surgical tools that the user can feel, thereby providing a presentation that the user would typically see and feel in a natural surgical environment. A hand rest (not illustrated) may also be provided to allow the user to stabilize his or her hands for the delicate manipulations required by the simulated surgical procedure.

Referring to Fig. 5, there is illustrated a top view of the user's perspective of one example of a simulated surgical procedure. In particular. Fig. 5 illustrates the surgical simulation of a surgical procedure known as anastomosis, which is the suturing of tube-like organs together. It is to be understood that anastomosis according to this invention, is defined as above to be the suturing of any tube-like organs such as. for example, blood vessels, the esophagus, ureters, bronchi, the colon, ducts and other organs found throughout the human body. In Fig. 5. the user 46 is handling a needle holder 22 and the forceps 20 in his or her hands that are connected to the force feedback devices (not illustrated). The user views a simulated needle holder 58. a simulated needle 60 and simulated thread 62 attached to the needle holder 22, and simulated forceps 56 attached to the forceps 20. The user can grasp a simulated tube 52, 54 with the simulated forceps and stabilize the tube while puncturing it with the simulated needle held in jaws of the simulated needle holder. The user can release and regrasp the needle with either tool. When the needle is released by the surgical tools it can either remain in a fixed position in the virtual body environment in one embodiment of the invention or can behave as a physically simulated object and fall under the simulated forces of gravity in another embodiment of the invention. Any interaction forces between the surgical tools and the needle are felt by the user with the surgical tools and the force feedback devices. The user sutures the tubes together by puncturing each tube in sequence, then pulling the suture material through the tube, then tightening the suture material as is typically done in the procedure of anastomosis. As will be discussed in detail below, the force of interaction between the simulated surgical tools and the simulated tubes are calculated by the simulation processor 12 (see Fig. 3) and conveyed to the user through the force feedback devices. In addition, the visual images of the interaction are calculated by the graphics processor 13 and displayed to the user via the 3-D graphics display 14. Accordingly, the surgical simulation system of Fig. 5 allows the user to practice the techniques of surgical anastomosis. It is to be appreciated that although Fig. 5 illustrates the surgical procedure of anastomosis, the invention is not to be limited to anastomosis and can include any surgical procedure and that all surgical procedures are intended to be within the term a simulated surgical procedure as defined by this invention. Referring to Fig. 6, the simulation system processor 12 of the system 10 of Fig. 4 also includes a surgical skill evaluation module 64 (See Fig. 8) that performs a surgical skill evaluation process 61 for measuring, recording, evaluating and reporting the surgical performance of the user of the simulated surgical procedure. In particular, the measured surgical tool positions, orientations, and forces that occur during the simulated surgical procedure are monitored by this module and stored (Step 63) in a programmable storage media associated with the system 10 of the invention. The stored information is used by the surgical skill evaluation module 64 for determining and providing a raw score of the user's performance (Step 65) based on physical data such as, for example, time information, force information, and distance information provided to the surgical skill evaluation module on lines 122, 124, 126, 128 and 130. The stored information is also compared to stored performance data of a population of practitioners in the field (Step 67) with experience ranging from inexperienced to expert (Step 69) who have also performed the simulated surgical procedure. The user's performance is also compared to a theoretical ideal performance (Step 71). All of this comparison data is used to determine a percentage ranking of the user's performance (Step 71).

The raw score and percentage ranking provide detailed information to the user that is useful for evaluation of the user's knowledge and skill and which, in turn, can be used for education, training, and/or accreditation of the user of the system. In particular, as will be discussed in greater detail below, the force, position, orientation, and velocity of the surgical tools such as the needle, needle holder, and forceps as well as the position and orientation of the elements of the body environment, can be used to evaluate the performance of the user. It is to be appreciated that the output of the information to the user can be provided in a plurality of different ways such as. for example, on the visual display in a graphical format, such as a plot 75 of needle force versus time, as illustrated in Fig. 7, by visually replaying both simulated surgeries to the user to view the techniques of the user and the "expert.^" and the like. All variations of this output are intended to be within the scope of the invention. With the surgical simulation system of the invention, the user can thus compare his or her performance to that of practitioners in the field and an "expert", and the information can be portrayed to the user in a plurality of different manners. Thus, the surgical simulation system of the invention can be used for education, training and accreditation of the user on a surgical procedure. Referring to Fig. 7. there is illustrated a display 112 of one example of a simulated surgical procedure that allows measurement of the user^'s surgical skill. According to this example of a simulated procedure, the tube has been provided with targets 63 for desired placement of needle sutures. The user is asked to place the needle through the tube at the desired target locations with the simulated needle holder 58 and needle 60. In particular, Fig. 7 illustrates the surgical simulation of an anastomosis procedure that can measure elements of surgical skill such as, for example, accuracy in the needle 60 placement, damage to the surface of the tube 52, tissue damage during needle penetration of the tube, curved needle suturing technique, peak forces applied to the tube, and accrued time required to perform the procedure. Fig. 7 includes the graph 75 of needle force versus time of the anastomosis procedure.

Fig. 8 illustrates an embodiment of the surgical skill evaluation module 64 of the surgical simulation system of the invention. The surgical skill evaluation module includes a needle accuracy measurement module 114, a surface damage measurement module 1 16. a tissue damage measurement module 118, a suturing technique measurement module 120. an accrued time measurement module and a peak force measurement module 119.. The surgical skill measurement module and in particular, the modules 1 14-120 within the surgical skill measurement module, use the following information: the position and orientation of the simulated surgical tools on line 122; the position and orientation of the simulated body environment on line 124; data indicating interaction between the simulated surgical tools and the simulated body environment on line 126; an indication of time on line 128; and data indicating forces of interaction between the simulated surgical tools and the simulated body environment on line 130. This information is used by the needle accuracy measurement module to measure the accuracy of the needle placement by the user, by the surface damage measurement module to measure the surface damage to the simulated body tube 52. prior to puncture by the needle, by the tissue damage measurement module to assess the tissue damage to the simulated tube once the needle has punctured the tube, by the suturing technique measurement module to provide an indication of the suturing technique of the user, by the accrued time measurement module to determine the time the user takes to perform the simulated task and by the peak force measurement module to measure the peak force applied by the user to the tube.

Referring to Fig. 9, there is illustrated a method performed by the needle accuracy measurement module 114 for determining the accuracy of needle placement by the user of the surgical simulation system. The needle accuracy measurement module measures the accuracy of the needle placement of the user by computing the distance between the desired needle puncture target 63 (see Fig. 7) and the actual placement of the needle and its puncture location on the simulated surgical tube 52. In particular, the needle accuracy measurement module determines a position of a tip of the simulated surgical needle from the data indicating the position of the simulated surgical tools on line 122 (Step 132), determines the position of the needle puncture target from the data indicating the position and orientation of the simulated body environment on line 124 (Step 134) and compares this information (Step 136) to determine a distance error between the position of the target and the actual position of the tip of the needle.

Referring now to Fig. 10. there is illustrated a method performed by the surface damage measurement module 116 of determining any surface damage to the simulated surgical tube by the user performing the simulated surgical procedure. In particular, the surface damage to the simulated surgical tube 52 is measured from the number of contacts made between the simulated surgical needle and the simulated surgical tube and from the information indicating the forces exerted by the user contacting the simulated surgical tube prior to puncturing the simulated surgical tube. In particular, the surface damage measurement module determines the interaction forces between the simulated surgical needle and the simulated surgical tube (Step 138) from the above-identified information. The module then determines whether the force is less than a puncture threshold of the simulated surgical tube (Step 140). If the force of interaction is not less than the puncture threshold. then the needle is simulated to puncture the simulated tube (Step 142). Alternatively, if the force is less than the puncture threshold of the simulated tube, then the force is multiplied by the interval of time over which the force is applied (Step 144), is added to a surface damage score, and the total is summed for a length of time the needle is in contact with the tube surface (Step 146) and not puncturing the tube surface, to determine an overall surface damage to the simulated surgical tube.

Referring now to Fig. 1 1. there is illustrated a method performed by the tissue damage measurement module 118 for assessing the tissue damage to the simulated body tube by measuring forces exerted between the simulated surgical needle and the simulated body tube. In particular, once the needle has punctured the tube, the interaction force between the simulated surgical needle and the simulated tube is determined from the above information

(Step 148) and compared (Step 150) to a minimum force required to advance the needle through the tube 152. The amount of force, or in other words, the excess force, above the minimum force required to advance the needle through the tube is provided on line 154. The excess force is summed over the time interval for which this excess force is present (Step 156) to provide an assessment of the amount of tissue damage to the simulated surgical tube.

Referring now to Fig. 12, there is illustrated a method performed by the suturing technique measurement module 120 measuring the suturing technique of the user of the surgical simulation system of the invention. In particular, the position and orientation of the needle is determined (Step 158) from the above-identified information and compared (Step 160) to an ideal relative orientation for advancing the simulated surgical needle through the simulated surgical tube provided on line 162. Any difference between the ideal relative position and orientation and the measured position and orientation of the simulated needle is output as an error value 164. The error value is summed per unit of needle length as the needle is advanced through the tube or in other words, for the entire stitch (Step 165) to provide a score value on line 167.

For each of the above measurement modules 1 14-120, these quantities of error or scores can be compared to a database of surgical skill scores to assess how the user^'s skill compares to other users of the system. It is to be appreciated that although the above description with respect to Figs. 7-12 illustrates a surgical testing procedure for the procedure of anastomosis, the invention is not to be limited to anastomosis and can include any surgical procedure and that all surgical procedures are intended to be within the scope of this invention. It is to be appreciated that access to detailed simulated physical measurements of the interaction between the simulated surgical tools and the simulated body environment for purposes of assessing skill of the user is an advantage of the present invention over the prior art that do not include, for example, dynamic simulation that allows detailed position, velocity, and force information of the simulated body environment and simulated surgical tools.

Referring now to Fig. 13, an embodiment of the simulation processor 12 of the surgical simulation system of Fig. 3 is illustrated. The processor includes a main memory 66 including a database for holding system data and/or data that may be provided by users of the system. For example, the data may be provided by users to the surgical simulation system via a keyboard (not illustrated) or by any other input means known in the art. The main memory may also include the stored performance data discussed above. A secondary memory 68 may also be provided for maintaining the integrity of the database of the system. The simulation processor 12 also includes a controller 70 for reading and writing of data from the database stored in the main memory and that controls the overall operation of the surgical simulation system and each of the modules to be described below. The processor further includes a display controller 72 that controls display of the simulated surgical tools and the simulated body environment on the 3-D graphics display 14 (see Fig. 3). Moreover, the processor includes: a 3-D graphics simulator 74 that renders, in real-time, a texture mapped simulated body environment and the simulated surgical tools; a force feedback interactor 76 that, as discussed above, applies forces to the surgical tools 20, 22 via the force feedback devices 16,

18 (see Figure 3); and a dynamic simulator 78 that simulates interaction of the simulated body environment and the surgical tools, as will be described in greater detail below.

Referring now to Fig. 14, an embodiment of the 3-D graphics simulator 74 is illustrated. The 3-D graphics simulator includes a 3-D object modeler 80 that includes 3-D object models of any of bones, joints, body tissues, organs, cartilage, muscle tissues, and the like of the simulated body environment as well as 3-D object models of the surgical tools. The 3-D object models include geometry used for visual display that may be comprised of polygonal models, NURB models, or other computer graphic geometry modeling techniques known to those skilled in the art. The 3-D object graphical models include parameters for size, texture, geometry and topology of the simulated objects of the body environment. The 3-

D graphics simulator also includes a texture mapping circuit 82 that applies texture mapping to the 3-D object models of the simulated body environment so that the simulated body environment and the components of the simulated body environment have a realistic appearance. Such texture mapping circuits are known to those of skill in the art. The 3-D graphics simulator further includes a lighting and point of view modeler 84 supporting lighting effects such as specular lighting, ambient lighting, and reflected lighting and supporting arbitrary views of the 3-D simulated environment through arbitrary positioning of the viewport. These lighting and point of view models are known to those of skill in the art. These models are used to simulate a realistic surgical simulation of the simulated body environment, the components of the simulated body environment and the simulated surgical tools. The 3-D graphics simulator also includes a deformation module 85 that efficiently modifies the geometry of elements of the simulated body environment in real-time to represent soft tissue deformation that occurs during interaction with the simulated surgical tools or other simulated objects. As will be discussed in detail below, the deformation module modifies the geometric models in the region near the application of an interaction force with the surgical tool or other element of the body environment. In one preferred embodiment of the surface deformation module 85, this interaction force is applied at one or more points on the simulated 3-D object such as, for example, the point at which the tube is grasped and pulled with the simulated forceps. In other words, for the anastomosis surgical procedure, the surface deformation module modifies the point on the tube that is grasped by the forceps to follow the forceps as the forceps are moved away from the tube. The tube geometry in the vicinity of the grasp point that represents the simulated body tube such as, for example, the vertices of a polygonal tube model, are modified according to a deformation function. As illustrated in Fig. 15 A, the deformation function adjusts a vertex position 166 of the tube model to a deformed vertex position 171 depending upon a magnitude M of the deformation of a grasp point to a deformed grasp point 170 and the distance d of the vertex 166 from the grasp point 168. The magnitude M and the distance d are used to determine a region of deformation 172 of the body tube, as illustrated in perspective view in Fig. 15B. This deformation module is efficient because it does not require modification of the entire surface geometry of the tube and the calculations for each point are simple. This deformation function can be applied to multiple points or regions of a geometric model. Referring now to Fig. 16, there is illustrated an embodiment of the dynamic simulator 78 of Fig. 13. The dynamic simulator provides for collision detection, interaction forces, and movement modeling of the simulated body environment and the components of the simulated body environment. In addition, the dynamic simulator provides for collision detection and interaction forces of the simulated surgical tools. Movement of the simulated surgical tools is determined from the measured position, orientation, and configuration information of the surgical tools mounted to the force feedback devices. The simulated components of the body environment are simulated to obey physical properties such as Newton's Laws. In particular, the simulated components of the body environment may be comprised of rigid links that are connected with flexible joints with one or more degrees of freedom. Rigid or deformable 3-D geometry may be attached to these links. When interaction forces are applied to the links of a simulated component of the body environment, the movement module computes its gross body movement and movement of its flexible joints according to Newton^'s Laws. As discussed above, the 3-D simulated components of the body environment have physical properties such as size, mass, inertia, and flexible joints according to the 3-D models. When the simulated components occupy a same physical space with each other or with the simulated surgical tools, the dynamic simulator simulates the components so they will collide and exchange interaction forces in equal and opposite directions according to Newton's Laws. Similarly, the dynamic simulator simulates the components so that when interaction forces are applied to simulated objects they move in response to those forces according to Newton^'s Laws. According to the embodiment of the dynamic simulator of the invention illustrated in

Fig. 16. this physically realistic behavior is simulated with three modules: a collision detection module 86, a contact force modeling module 88 and a movement modeling module 90.

The dynamic simulator module further includes a simulated object kinematics module 91 which determines the position and orientation of any part of a simulated object from knowledge of the object's shape and the position of the object's joints or degrees of freedom. The dynamic simulator module also includes a surgical tool kinematics module 93 which determines the position and orientation of any part of a simulated surgical tool. The surgical tool kinematics model receives measurements of the position of the force feedback device on line 174, information about the structure of the force feedback device on line 176, measurements from the sensors on the surgical tools on line 178, and information about the structure of the surgical tool on line 180 to compute the position and orientation of any part of the surgical tool. Since the surgical tool kinematics do not depend upon the simulated body model, the surgical tool kinematics module can determine the position and orientation of any part of the surgical tool independent of the simulated body model.

The collision detection module 88 detects a collision between any of two simulated body components, a simulated body component and the simulated body environment, or a simulated body component and a simulated surgical tool or two or more simulated surgical tools (hereinafter "simulated objects"). The collision detection module checks the distance between features of simulated objects to check for interpenetration of 3-D geometric representations of two simulated objects. These collision detection techniques are generally known to those of skill in the art. If the two simulated objects are found to be interpenetrating, then the collision detection module determines the surface features of the simulated objects that are colliding and the amount of interpenetration is calculated.

When two simulated 3-D objects are found to be in contact, then the contact force modeling module 88 will compute interaction forces between the simulated objects in order to simulate interaction of the two simulated objects. In particular, equal and opposite contact forces are applied to each simulated object in accordance with Newton's Third Law. The contact force modeling module determines the interaction forces based upon the relative position and velocity of points on the surface of each simulated object. The position and velocity information is provided by the surgical tools kinematics module 93 and the simulated organ kinematics module 91. The contact forces are computed with force models that combine the relative position and velocity information with spring elements, dissipative elements, and frictional elements as is generally known to those of skill in the art. The spring elements determined by the force models are a function of the relative position of the contacting surfaces of the simulated 3-D objects. The dissipative elements are a function of relative velocity of the contacting surfaces. According to this embodiment of the dynamic simulator of the invention, the net forces due to the spring and dissipative elements are constrained to satisfy friction models such as, for example, Coulomb friction, static friction, and kinetic friction. The friction models provide for slipping of one contacting surface of a 3- D simulated object relative to another contacting surface of the other 3-D object. The friction models constrain the contact forces such that they obey certain relationships between the simulated objects. These friction models are generally known to one of skill in the art. The contact force modeling module then provides an output to the movement modeling module 90 indicating the contact forces that result from the simulated contact between the simulated objects of the simulated body environment.

The movement modeling module 90 determines movement of the simulated objects in response to forces provided by the contact force modeling module 88, that arise from contact between any of the simulated objects and/or forces due to gravity. As discussed above, the simulated objects have physical attributes such as, for example, mass, inertia, energy and momentum. Accordingly, the simulated objects move realistically in response to contact forces; this movement is determined by the movement modeling module according to Newton's Second Law of Motion. The movement modeling module includes the physical description of the simulated objects such as each object's mass, inertia, size and its joints or degrees of freedom. The movement modeling module also includes an acceleration modeling module 92 that determines the acceleration of the simulated objects and degrees of freedom, such as, for example, translation and rotation of the simulated objects in response to the application of the above-described external forces and or a lack of such external forces. In particular, referring to Fig. 17, there is illustrated a method of determining the movement of the simulated objects according to the invention. The movement modeling module receives the contact force information (Step 184) from the contact force modeling module. The movement modeling module computes the acceleration (Step 186) of the simulated objects. The acceleration of objects is integrated with time to determine a velocity (Step 188) and a position (Step 190) of the simulated objects through a numerical integration process generally known to one of skill in the art.

As discussed above with respect to Fig. 5, one embodiment of the surgical simulation system 10 of the invention is specialized for learning and practicing the anastomosis of biological vessels. The real-time, dynamic performance of the embodiment of the invention simulating anastomosis is provided by an efficient technique of the invention for physical simulation of the biological vessels including their dynamic movement, contact detection, deformation and contact force modeling. Typical flexible tissue simulations known to those of skill in the art employ high degree of freedom finite element models to simulate continuously deformable objects like biological tissue. However, a problem with these models is that they are not efficient. In particular, the complexity of these finite element models prevents them from being used in a real-time, dynamic simulation environment such as the surgical simulation system of the invention. One reason smoothly deformable models typically used by those of skill in the art are inefficient is because they require high degree-of- freedom models in order to create smooth geometry. However, high-degree-of-freedom models are computationally expensive for the dynamics simulators to model. In contrast, the tube models of the present invention incorporate low-degree-of-freedom geometric models that deform smoothly and low degree-of-freedom dynamic models that capture physically realistic behavior. According to this embodiment of the invention, an embodiment of the dynamic simulator 78' is used as is illustrated in Fig. 18. It is to be appreciated that any common elements of the modules of the dynamic simulator 78' of Fig. 18 to that of the dynamic simulator 78 of Fig. 16 are illustrated with like reference numerals and that the description of the elements with respect to Fig. 16 also applies to Fig. 18.

One embodiment of a tube model is based on spline functions and is illustrated in Fig. 19. A spline 192 is the centerline of the tube model. The spline is defined by the placement of four control points 196. An interior wall of the tube is defined by a radius Ri from the spline. An exterior wall of the tube is defined by a radius Ro from the spline. Deformation to the tube model is applied with the surface deformation module 85 of Fig. 14 as illustrated in

Figs. 15A-15B. Movement of the control points results in movement of the spline. If the control points are attached to dynamic models that move according to physics, the tube will move with characteristic physical traits.

Fig. 20 illustrates another embodiment of a simple dynamic tube model 195. Two control points 196 of the spline 192 are fixed in space. Two other control points 198 are attached to a rigid body 200 that has a mass and inertia that is anchored with compliant linear and torsional springs 202 to the simulated body environment. Translation and rotation of the rigid body results in movement of the control points which produces smooth translation and rotation of the tube. Simulated surgical tool forces applied to the tube surface are applied to the rigid body allowing the tube to move in response to interaction forces.

Fig. 21 illustrates a second simple dynamic tube model 204 according to the invention. This tube model uses a double link pendulum having a first link 206 and a second link 208 with rotational joints 210. The links of the pendulum have mass and inertia. Each link of the pendulum has two control points 194 of the spline attached to it. Torsional springs 212 at each joint of the pendulum provides inherent compliance along the length of the tube.

Interaction forces applied to the surface of the tube are applied to the double pendulum allowing the tube to respond to interaction forces in a realistic way. Having illustrated a number of simple tube models, it is to be appreciated that other variants of these tube models that may involve additional splines or different simplified models are intended to be within the scope of the invention.

Referring again to Fig. 18. the embodiment of the dynamic simulator 78' includes a rigid body translation and rotation modeling module 94 having rigid body object models of geometry, physical behavior and feel of the tube-like organs, as discussed above. The rigid body translation and rotation module includes models that are layered to adapt to different modes of tube deformation. The models include parameters such as tube length, width, tissue thickness, mass, inertia, compliance, puncture forces, surface friction, surface texture and surface coloration. According to this embodiment of the invention, the output of the dynamic simulator is a realistic-looking simulated tube-like object that can be dynamically simulated in a real time environment.

The dynamic simulator 78' of Fig. 18 also includes a collision detection module 100 that determines any interaction of the simulated tubes and the simulated surgical tools. The collision detection module detects collision between a point or line segment on the surgical tool and the tube surface. The module finds the point on the spline which is closest to the tool then calculates the distance. If the distance is less than or equal to the radius of the tube at that point, then contact is confirmed. Non-circular cross sections of the tube can be handled by considering variable radius of the tube. The dynamic simulator 78' further includes a contact force modeling module 88 that computes interaction forces between the simulated objects and a movement modeling module 90 that determines movement of the simulated tube in response to forces provided by the contact force modeling module. The movement modeling module may be a surface deformation modeling module 98 that simulates deformation of the tube as a result of contact, for example, with the simulated surgical tools. A parameterized deformation function as described above will distort the tube surface either outward or inward based upon a position of a tube center line and the simulated surgical tool. According to the surface deformation modeling module, the polygons describing the tube surface are modified to reveal a local geometry change in the tube surface which will change with simulated surgical tool movement and/or with the simulated movement. It is to be appreciated that the surface deformation module 98 according to this embodiment of the invention, although illustrated with respect to flexible tubes, can also be extended to all flexible organ simulations and is intended to be within the scope of the invention. If movement of a flexible organ is simulated using the simple rigid body models described above and surface deformation is simulated using the above-described deformation model, the dynamic simulator will simulate a dynamic, flexible and deformable tube-like organ. It is also to be appreciated that other geometry change models can be implemented according to the invention to emulate reactions to movement and deformations such as incisions, probing and puncture of the simulated tube-like organs and that such simulations are intended to be within the scope of the invention. Therefore, according to the invention, a surgical simulation system and method is provided for learning, practicing, experiencing and evaluating surgical procedures. With the method and system of the invention, the need for physical models or live patients is eliminated and a life-like simulation is provided such that the user sees and hears a life-like simulated surgical environment and dynamic situations in real-time using 3-D computer generated graphics and sound, and such that the user feels the interaction with the simulated body environment and simulated surgical tools.

An advantage of the surgical simulation system and method of the invention is that it will help to improve medical education by providing a plurality of life-like surgical situations to the user to experience, practice and perfect. The system and method may be used to control and standardize training regimen. With the system and method of the invention, students can learn by practicing. In particular, students can be presented with lessons that are commensurate with their skill level and students can learn to adapt procedures for a range of anatomical variations and surgical conditions. Moreover, students will be able to repeat the simulated surgical procedures until they master them without fear of any harm to a patient. The surgical simulation system and method of the invention also makes it possible to objectively evaluate the surgical skills of a student. The detailed information provided by the system and method such as, for example, position, velocity and force information makes it possible to measure the tool accuracy, tissue damage, and surgical techniques of the student. The students will also be able to review their performance and compare it to that of "experts" in the field who have performed the same procedure. Moreover, by using the recorded performance of the "expert" as a standard, the surgical simulation system and method of the invention further enhances the teaching capacity of experts beyond the one-to-one apprenticeship that is the standard today.

Another advantage of the surgical simulation system and method of the invention is that it can also be used with traditional training techniques to improve the quality and reduce the costs of surgical education by insuring that students are prepared to make the best possible use of valuable time that is provided to the students in the operating room.

Still other advantages of the surgical simulation system and apparatus of the invention include: that surgical training can be provided without the use of human patients, hospital space or animals; the surgical simulation system and method will help develop and train the physical motor skills of the student, the perceptual tasks of the student and the cognitive decision-making of the student, and the surgical training system and method will allow medical students to practice routine procedures, encounter patients with rare medical conditions or unexpected complications and to practice techniques to handle each of these situations. Accordingly, in contrast to accepted practice where only experienced surgeons are exposed to such a wide range of conditions and complications that can occur, the surgical simulation system and method of the invention will provide a wide range of conditions and complications to any user. In addition to the training provided by the surgical simulation system and method of the invention, the system D and method of the invention can also be used to assess the surgical performance of the student and play a roll in certification of the student for surgical procedures. Having thus described several embodiments of the invention, various alterations, modifications, and improvements will readily occur to those skilled in the art. Such alterations, modifications, and improvements are intended to be part of this disclosure, and are intended to be within the spirit and scope of the invention. Accordingly, the foregoing description is by way of example only and the invention is limited only as defined in the following claims and the equivalents thereto.

What is claimed is:

Claims

1. An interactive, real-time simulated surgical simulation system comprising: a body environment simulator that simulates a body environment having components including any of bones, joints, tissues and organs; a surgical tool simulator that simulates at least one surgical tool within the body environment, the simulated surgical tool being coupled to a physical tool that is manipulated by a user of the surgical simulation system; a visual display device for displaying the body environment and the simulated surgical tool; an interaction simulator that simulates in real-time interaction between either of the body environment and the components of the body environment and between the simulated surgical tool, and for providing the simulated interaction on the visual display device; and a force presenting device that presents simulated forces of the interaction at the physical tool.

2. The interactive, real-time surgical system as claimed in claim 1, wherein the simulated interaction includes any of a collision, an exchange of force or energy, movement, deformation, and a response to either of the collision and the exchange of force or energy between either of the body environment and the components of the virtual body environment and between the simulated surgical tool.

3. The interactive, real-time surgical system as claimed in claim 2, wherein the simulated interaction includes any of tearing, puncturing, an incision, and bleeding of either of the body environment and the components of the body environment.

4. The interactive, real-time surgical system as claimed in claim 1, further comprising: a 3D graphics simulator including 3D geometric object models and components that simulates the body environment, the components of the body environment and the surgical tool in 3-D.

5. The interactive, real-time surgical system as claimed in claim 4, wherein the 3D graphics simulator includes a texture mapping circuit that provides texture to the body environment and the components of the body environment so as to represent realistic body environment and components.

6. The interactive, real-time surgical system as claimed in claim 1, wherein the body environment simulator and the surgical tool simulator include lighting models and point of view models that create arbitrary views of the body environment and simulated surgical tool.

7. The interactive, real-time surgical simulation system as claimed in claim 1 , wherein the force presenting device includes an active force feedback device coupled to the physical tool.

8. The interactive, real-time surgical system as claimed in claim 1, further comprising a surgical tool kinematics module that determines a position, orientation, and configuration of the simulated surgical tool that is independent of the body environment.

9. The interactive, real-time simulation system as claimed in claim 1 , wherein the interaction simulator includes a collision detector that determines whether any collision occurs between the simulated surgical tool and either of the body environment and the components of the body environment.

10. The interactive, real-time surgical simulation system as claimed in claim 9. wherein the interaction simulator further includes a contact force modeling module that determines an exchange of force between the simulated surgical tool and either of the body environment and the components of the body environment.

11. The interactive, real-time surgical simulation system as claimed in claim 2, wherein the interaction simulator includes a movement modeling module and a surface deformation module that determine movement and deformation of any of the simulated surgical tool, the body environment, and the components of the body environment in response to any of the exchange of force or energy between the simulated surgical tool, the body environment and the components of the body environment, forces due to gravity, and a lack of any force.

12. The interactive, real-time surgical simulation system as claimed in claim 11, wherein the movement modeling module distorts a surface of a component of the body environment to provide a deformed component according to a shape function and according to a relative position of the simulated surgical tool to the component of the body environment.

13. The interactive, real-time surgical simulation system as claimed in claim 11 , wherein the components of the body environment are simulated with any of size, mass, compliance, friction, texture, geometry, and topology information.

14. The interactive, real-time surgical simulation system as claimed in claim 13, wherein the movement modeling module further includes an acceleration module that determines an acceleration of any of the body environment and the components of the body environment in response to the contact between the simulated surgical tool and either of the body environment and the components of the body environment.

15. The interactive, real-time surgical simulation system as claimed in claim 1, wherein the body environment simulator includes a plurality of models of tube-like organs each having any of length, width, thickness, mass, inertia, compliance, puncture resistance, friction, surface texture and surface color information.

16. The interactive, real-time simulation system as claimed in claim 15, wherein the interaction simulator simulates interaction between the tube-like organs such that the organs can be sewed together with the at least one simulated surgical tool, a simulated needle and a simulated thread.

17. The interactive, real-time simulation system as claimed in claim 16, wherein the tubelike organs are simulated with targets identified for placement of sutures for the anastomosis.

18. The interactive, real-time surgical simulation system as claimed in claim 1, further comprising a surgical skill module that records data, including any of surgical tool position and orientation, body environment and body components position and orientation, interaction forces, and an elapsed time of a simulated medical procedure by the user.

19. The interactive, real-time surgical simulation system as claimed in claim 18, wherein the surgical skill measurement module includes idealized performance data indicative of an ideal performance of the simulated medical procedure.

20. The interactive, real-time surgical simulation system as claimed in claim 19, wherein the surgical skill measurement module compares the idealized performance data with the data of the simulated medical procedure to provide comparison data that is presented to the user via the visual display device.

21. The interactive, real-time surgical simulation system as claimed in claim 20, wherein the surgical skill measurement module analyzes the comparison data to provide an assessment of the simulated surgical procedure.

22. A method of providing an interactive, real-time surgical simulation system, comprising the steps of: simulating a body environment having components including any of bones, joints, tissues and organs; simulating at least one surgical tool within the body environment, the simulated surgical tool being coupled to a physical tool that is manipulated by a user of the surgical simulation system; displaying the body environment and the simulated surgical tool; simulating in real-time interaction between either of the body environment and the components of the body environment and between the simulated surgical tool; and presenting simulated forces interaction at the physical tool.

23. The method of providing the interactive, real-time surgical system of claim 22, wherein the step of simulating the interaction includes simulating any of a collision, an exchange of force or energy, movement, deformation, and a response to either of the collision and the exchange of force of energy between either of the body environment and the components of the body environment and between the simulated surgical tool.

24. The method of providing the interactive, real-time surgical simulation system of claim 23, wherein the step of simulating the interaction includes simulating any of tearing, puncturing, an incision, and bleeding of either of the body environment and the components of the body environment.

25. The method of providing the interactive, real-time surgical simulation system as claimed in claim 22, wherein the step of simulating the body environment and the step of simulating the surgical tool include simulating the body environment, the components of the body environment and the surgical tool in 3D.

26. The method of providing the interactive, real-time surgical simulation system as claimed in claim 25, wherein the step of simulating the body environment includes texture mapping the body environment and the components of the body environment so as to represent a realistic body environment and components.

27. The method of providing the interactive, real-time surgical simulation system as claimed in claim 22, wherein the step of simulating the body environment and the surgical tool include creating arbitrary views of the body environment and the simulated surgical tool.

28. The method of providing the interactive, real-time surgical simulation system as claimed in claim 22, further comprising the step of computing a position, orientation, and configuration of the simulated surgical tool that is independent of the body environment.

29. The method of providing the interactive, real-time simulation system as claimed in claim 23, further comprising the step of detecting any collision and exchange of force or energy between the simulated surgical tool and either of the body environment and the components of the body environment.

30. The method of providing the interactive, real-time surgical simulation system as claimed in claim 29, further comprising the step of computing the exchange of force between the simulated surgical tool and either of the body environment and the components of the body environment.

31. The method of providing the interactive, real-time surgical simulation system as claimed in claim 23, further comprising the step of computing any movement and deformation of any of the simulated surgical tool, the body environment, and the components of the body environment in response to any of the exchange of force or energy between the simulated surgical tool, the body environment and the components of the body environment, forces due to gravity, and a lack of any force.

32. The method of providing the interactive, real-time surgical simulation system as claimed in claim 31, wherein the step of computing includes distorting a surface of the components of the body environment according to a shape function and according to a relative position of the simulated surgical tool to an undeformed component of the body environment.

33. The method of providing the interactive, real-time surgical simulation system as claimed in claim 31 , wherein the step of simulating the body environment includes simulating the body environment with any of size, mass, compliance, friction, texture, geometry, and topology information.

34. The method of providing the interactive, real-time surgical simulation system as claimed in claim 33, wherein the step of computing includes computing an acceleration of any of the body environment and the components of the body environment in response to the contact between the simulated surgical tool and either of the body environment and the components of the body environment.

35. The method of providing the interactive, real-time surgical simulation system as claimed in claim 22, wherein the step of simulating the body environment includes simulating a plurality of tube-like organs each having any of length, width, thickness, mass, inertia, compliance, puncture resistance, friction, surface texture and surface color information.

36. The method of providing the interactive, real-time surgical simulation system as claimed in claim 35, further comprising the step of simulating anastomosis of the tube-like organs, wherein the tube-like organs can be sewed together with the at least one simulated surgical tool, a simulated needle and a simulated thread.

37. The method of providing the interactive, real-time surgical simulation system as claimed in claim 22, further comprising the step of recording data, including any of surgical tool position and orientation, body environment and body components position and orientation, interaction forces, and an elapsed time of a simulated medical procedure by the user.

38. The method of providing the interactive, real-time surgical simulation system as claimed in claim 37, further comprising the step of comparing idealized performance data with the data of the simulated medical procedure to provide comparison data that is presented to the user.

39. The method of providing the interactive, real-time surgical simulation system as claimed in claim 38, further comprising the step of analyzing the comparison data to provide an assessment of the simulated surgical procedure.