WO2019180747A1 - Systèmes et procédés pour l'obtention de modèles d'instruments propres à un patient - Google Patents

Systèmes et procédés pour l'obtention de modèles d'instruments propres à un patient Download PDF

Info

Publication number
WO2019180747A1
WO2019180747A1 PCT/IN2019/050233 IN2019050233W WO2019180747A1 WO 2019180747 A1 WO2019180747 A1 WO 2019180747A1 IN 2019050233 W IN2019050233 W IN 2019050233W WO 2019180747 A1 WO2019180747 A1 WO 2019180747A1
Authority
WO
WIPO (PCT)
Prior art keywords
bone
silhouette
mesh
template
positions
Prior art date
Application number
PCT/IN2019/050233
Other languages
English (en)
Inventor
Vikas KARADE
Amit Maurya
Original Assignee
Karade Vikas
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Karade Vikas filed Critical Karade Vikas
Publication of WO2019180747A1 publication Critical patent/WO2019180747A1/fr

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B17/00—Surgical instruments, devices or methods, e.g. tourniquets
    • A61B17/14—Surgical saws ; Accessories therefor
    • A61B17/15—Guides therefor
    • A61B17/154—Guides therefor for preparing bone for knee prosthesis
    • A61B17/155—Cutting femur
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B17/00—Surgical instruments, devices or methods, e.g. tourniquets
    • A61B17/14—Surgical saws ; Accessories therefor
    • A61B17/15—Guides therefor
    • A61B17/151—Guides therefor for corrective osteotomy
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B17/00—Surgical instruments, devices or methods, e.g. tourniquets
    • A61B17/56—Surgical instruments or methods for treatment of bones or joints; Devices specially adapted therefor
    • A61B2017/568—Surgical instruments or methods for treatment of bones or joints; Devices specially adapted therefor produced with shape and dimensions specific for an individual patient
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
    • A61B2034/101—Computer-aided simulation of surgical operations
    • A61B2034/102—Modelling of surgical devices, implants or prosthesis
    • A61B2034/104—Modelling the effect of the tool, e.g. the effect of an implanted prosthesis or for predicting the effect of ablation or burring
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
    • A61B2034/101—Computer-aided simulation of surgical operations
    • A61B2034/105—Modelling of the patient, e.g. for ligaments or bones

Definitions

  • This invention relates to the field of biomedical engineering.
  • this invention to systems and methods for obtaining patient specific instrument designs.
  • Surgical planning is a preoperative method of visualising a surgical intervention, to set out the surgical steps and bone segment navigation in the context of computer assisted surgery. Surgical planning is important in orthopedic surgery,
  • Some orthopedic surgeries like knee or hip replacement, include cutting or drilling on an irregular- shaped a bone. Performance and accuracy of such surgeries improves if the surgery is planned pre-operatively. Surgeons are trained to use conventional 2D image data to prepare for their complex procedures. Such planning may be made from X-ray images of CT data sets or the like. CT data sets are large compared to X-ray images. Hard copies of X-ray images of the particular region of the patient's body for operation, such as a knee or hip-joint, or digital X- ray images on a PC based, can be used for 2D operational planning.
  • Example embodiments include computer systems for transforming 2D anatomical X-ray images into 3D renderings for surgical preparation through example methods. Such methods include taking x-ray image of body part to be converted to 3D and determining a camera model of the x-ray image. For example, spatial values of the X-ray source and body part may indicate the camera model. A contour of the body part is extracted from the X-ray and analyzed based on its anatomical regions. Each region is assigned 2D anatomical values in the contour.
  • a separate 3D template for the body part is then modified to match the 2D X-ray images by extracting silhouette vertices from the 3D template and their projections, according to the camera model and how those features are initially aligned in the template.
  • the template can then be aligned with the x-ray image and projected on an image plane for the appropriate camera model to obtain a 2D projection model.
  • the template is then modified to match the 2D anatomical values by comparing the 2D projection with the corresponding identified anatomical values.
  • a best matching point on the contour, for each extracted silhouette vertex projection is identified between the 2D projection and contour.
  • the resulting matching points are then back projected based on camera model to form a back projected ray with target positions that are closest to a corresponding silhouette vertex.
  • the 3D template can then be deformed so that its silhouette vertices match the target positions, resulting in a 3D image that corresponds to the 2D X-ray image.
  • a limitation of the prior art is that input images required to be orthogonally oriented. Therefore, there is a need for a system and method which can provide a deformed 3D reconstructed image which is a pre-cursory input towards a final 3D reconstructed image. According to the prior art, the following steps refer to a deformation methodology:
  • target position being a position, on each of said back projected rays, that is closest to a corresponding silhouette vertex
  • the following steps refer to obtaining a method for obtaining a 3-dimensional image using at least one conventional 2-dimensional X- ray image, the method comprising:
  • anatomical region includes distinct anatomical regions; - identifying anatomical values of the contour, wherein the anatomical values are 2-dimensional anatomical values from the distinct anatomical regions;
  • an object of the invention is to provide systems and methods for obtaining patient specific instrument (PSI) which guides surgeon to perform operations on bone like resection or drilling
  • PSI patient specific instrument
  • an object of the invention is to provide systems and methods for automatic design of the PSI using the 2D X-ray based 3D bone model, for a region (proximal, distal, or mid-shaft) of a bone.
  • a method for obtaining patient specific instrument designs for bones comprising pegs, guiding slots, and a frame, said method comprising the steps of:
  • said silhouette positions set being selected from at least an AP silhouette positions set and / or an ML silhouette positions set, said silhouette sets being divided into at least four silhouette subsets, said silhouette subsets being selected from a group consisting of at least a first silhouette subset (SAP-X), at least a second silhouette subset (SAP-Z), at least a third silhouette subset (SML-Y), and at least a fourth silhouette subset (SML-Z);
  • This method may use 2-dimensional X-ray based 3 -dimensional bone model.
  • said step of computing silhouette positions comprising a step of computing silhouette positions, on said bone mesh, with respect to an X-ray source, defined in a camera model, in such a way that each silhouette position lies on an edge of said bone mesh in such a way that a line from the position of the X- ray source to the silhouette position does not pass through any face of said bone mesh.
  • said imaging source is an X-ray source.
  • said step of selection of peg type comprising a step of selection of peg type selected from a group consisting of at least a spherical peg type, at least a cylindrical peg type, and a cuboidal peg type, characterised in that, any of said pegs being configured to make point contact with a surface of the bone at a landing point.
  • said AP silhouette positions set being positions of the set of points lying within a user-defined distance around all the silhouette positions calculated with respect to an AP X-ray source.
  • said ML silhouette positions set being positions of the set of points lying within a user-defined distance around all the silhouette positions calculated with respect to an ML X-ray source.
  • said first silhouette subset being defined as a subset of the AP silhouette positions lying on part of the mesh faces which is visible from positive and negative direction of X-axis of anatomical co-ordinate system (ACS) and lying on the part of said bone mesh where the surgery is performed.
  • ACS anatomical co-ordinate system
  • said second silhouette subset being defined as a subset of the AP silhouette positions lying on part of the mesh faces which is visible from positive and negative direction of Z-axis of anatomical co-ordinate system (ACS) and lying on the part of said bone mesh where the surgery is performed.
  • ACS anatomical co-ordinate system
  • said third silhouette subset being defined as a subset of the ML silhouette positions lying on part of the mesh faces which is visible from positive and negative direction of Y-axis of anatomical co-ordinate system (ACS) and lying on the part of said bone mesh where the surgery is performed.
  • ACS anatomical co-ordinate system
  • said fourth silhouette subset being defined as a subset of the ML silhouette positions lying on part of the mesh faces which is visible from positive and negative direction of Z-axis of anatomical co-ordinate system (ACS) and lying on the part of said bone mesh where the surgery is performed.
  • ACS anatomical co-ordinate system
  • said guiding slots act as a guide for either drilling holes or cutting, the position and orientation of a guiding slot is fixed with respect to the position and orientation of a user-defined bone cutting plane / drilling plane and hence with respect to position of landing points on said bone mesh.
  • said frame connects all pegs and guiding slots together to form a complete patient specific instrument.
  • said step of selecting landing points comprising a step of selecting position of landing points from a subset of silhouette positions on said bone surface in such a way that it satisfies a criterion that the position of said selected landing points restrict 2 translational degrees and all 3 rotational degrees of freedom, of said patient specific instrument, about 3 axis of Anatomical Coordinate System (ACS), when said patient specific instrument is held on bone by a surgeon and restricting the remaining 1 translational degree of freedom.
  • ACS Anatomical Coordinate System
  • said step of selecting landing points comprising a step of selecting position of landing points within a subset in such a way that at least 2 selected point positions in that particular subset are spaced apart at least by an appropriate distance calculated for that subset.
  • said step of selecting landing points comprising a step of selecting at least 5 landing positions from all silhouette positions set, characterised in that:
  • first 3 silhouette positions being selected from a first silhouette subset (SAP- X) or a second silhouette subset (SAP-Z);
  • said step of selecting landing points comprising a step of selecting at least 5 landing positions from all silhouette positions set, characterised in that:
  • first 3 silhouette positions being selected from a third silhouette subset (SML-Y) or a fourth silhouette subset (SML-Z); and - next 2 silhouette positions being selected from a first silhouette subset (SAP- X) or a second silhouette subset (SAP-Z).
  • said step of selection of peg type based on landing position, comprising a step of selection of peg type selected from a group consisting of at least a spherical peg type, at least a cylindrical peg type, and a cuboidal peg type.
  • said step of selection of peg type, based on landing position comprising a step of selection of a spherical peg type if landing position is included in both first and second silhouette subset along with third and fourth silhouette subset.
  • said step of selection of peg type, based on landing position comprising a step of selection of a spherical peg type if landing position lies in the undercut or concave surface region of the bone mesh.
  • said step of selection of peg type, based on landing position comprising a step of selection of a cylindrical peg type, if the landing position is included in one of first silhouette subset, second silhouette subset, third silhouette subset, or fourth silhouette subset.
  • said step of selection of peg type, based on landing position comprising a step of selection of a cuboidal peg type, if the landing position lies in a near- spherical region of said bone mesh.
  • said step of determining orientation and position of said selected pegs comprising a step of determining a position, for a spherical peg, such that its centre is positioned at a distance, which is equal to the radius of the spherical peg, from said landing position in the direction of normal of the corresponding face of said bone mesh.
  • said step of determining orientation and position of said selected pegs comprising a step of determining a position, for a cylindrical peg, such that its centre is positioned at a distance, which is equal to the radius of the cylindrical peg, from said landing position in the direction of normal of the corresponding face of said bone mesh.
  • said step of determining orientation and position of said selected pegs comprising a step of determining an orientation, for a cylindrical peg, such that its axis is parallel to a line connecting to the landing position and a corresponding X-ray source.
  • said step of determining orientation and position of said selected pegs comprising a step of determining a position, for a cuboidal peg, such that its centre is positioned at a distance, which is equal to the half the width of the cuboidal peg, from said landing position in the direction of normal of the corresponding face of said bone mesh.
  • said step of determining orientation and position of said selected pegs comprising a step of determining an orientation, for a cuboidal peg, such that its axis is parallel to a line connecting to the landing position and a corresponding X-ray source.
  • said guiding-slot is a 3D mesh, with vertices and faces, with cylindrical guiding for drilling hole or resection, the orientation of the axis of said cylinder and position of said slot(s) is same as the orientation and position of a user defined cutting plane or drill holes with respect to said bone mesh.
  • said guiding-slot is a 3D mesh, with vertices and faces, with cuboidal guiding for drilling hole or resection, the orientation of the axis of said cuboidal and position of said slot(s) is same as the orientation and position of a user defined cutting plane or drill holes with respect to said bone mesh.
  • said step of generating a frame mesh comprising a step of uniting each of said meshes of said pegs, said guiding slots, and said frame.
  • said step of generating a frame mesh comprising a step of uniting each of said meshes of said pegs, said guiding slots, and said frame by means of constructive solid geometry.
  • said patient specific instrument template for said bone is a 3- dimensional mesh (with vertices and faces) with triangular elements, said patient specific instrument template design being specific to a region of a bone.
  • said patient specific instrument template for said bone is a 3-dimensional mesh (with vertices and faces) with triangular elements, said patient specific instrument template design being specific to a region of a bone, said template being divided into following three portions:
  • - pegs’ portion configured to be protruded structures of said patient specific instrument that come in direct contact with a bone when said patient specific instrument is placed on surface of said bone;
  • said method comprising the following steps in order to design a patient specific instrument for a region of a bone reconstructed using 2- dimensional X-ray images:
  • said anatomical landmarks are vertices of deformed bone which represent unique bony features, said landmarks being calculated based on standard directions of a bone’s Anatomical Coordinate system (ACS), as either extreme points of said 3 -dimensional bone model in the directions of Anatomical Coordinate system or fitting standard geometrical shapes on segmented regions of said 3 -dimensional bone model.
  • ACS Anatomical Coordinate system
  • said anatomical landmarks being calculated for a region for which said patient specific instrument is to be placed.
  • said anatomical parameters defining deformity in said bone being calculated along standard directions
  • said patient specific instrument comprising a template having certain landmarks corresponding to anatomical landmarks of a bone region where said patient specific instrument will be placed.
  • said patient specific instrument template is aligned (rotated and translated) in a coordinate system of said 3 -dimensional bone model in such a way that relative orientation of said anatomical landmarks of said bone region matches relative orientation of corresponding landmarks of said patient specific instrument template.
  • said patient specific instrument template is translated along an average normal direction of a bone region in such a way that there is no intersection between said patient specific instrument template model and said bone model and wherein average normal direction of a region is the average of all the coordinates of the normal vectors of the faces of a region.
  • said aligned patient specific instrument template is deformed in such a way that the pegs only are translated into certain positions with respect to the bone region while preserving the topology as much as possible.
  • deformation of said patient specific instrument mesh is performed using Laplacian surface deformation (LSD).
  • LSD Laplacian surface deformation
  • deformation of said patient specific instrument mesh is performed using Laplacian surface deformation (LSD) wherein said Laplacian surface deformation smoothly deforms a mesh while bringing a few selected anchor vertices of the mesh to respective target positions (positional constraints) and maintaining the inter-vertices positional relationship (Laplacian constraints) described by Laplacian coordinates.
  • LSD Laplacian surface deformation
  • vertices of said pegs’ portion of said patient specific instrument template are selected as anchor vertices wherein target positions for said anchor vertices are calculated based on silhouette vertices of said bone region, determined using respective X-ray image and its calibration parameters.
  • a silhouette vertex of the bone region is selected for each peg portion of said patient specific instrument template based on its closest proximity to the centroid position of the vertices of the peg portion, translation parameters (tx, ty, tz) being calculated as the difference in the coordinates of the selected silhouette vertex and the centroid of the peg region, the target positions mentioned above are calculated as those positions of the vertices of the peg region if they would be translated by the translation parameters.
  • guiding-slot is a 3-dimensional mesh (with vertices and faces) with cylindrical (guiding for drilling hole) or rectangular (guiding for cutting plane) slot within.
  • FIG. 1 is an illustration of a schematic block diagram of an example embodiment system.
  • FIG. 2 is an illustration of a camera model source positioning.
  • FIG. 3A is an illustration of anatomical regions for femur and tibia.
  • FIG. 3B is an illustration of anatomical landmarks and the anatomical parameters for femur and tibia.
  • FIG. 3 C is an illustration of anatomical regions corresponding to the regions distinguished in the contour of the X-ray image.
  • FIG. 3D is an illustration of anatomical landmarks identified based on anatomical regions.
  • FIG. 4 is an illustration of triangulation of projected points, meshing after putting constraints and the outer contour calculation.
  • FIG. 5 is an illustration of femur and tibia images wherein with corresponding transformations to the template.
  • FIG. 6 is an illustration of the template model before and after the alignment.
  • FIG. 7 is an illustration of template deformation.
  • FIG. 8 is an illustration of deformation for local matching.
  • FIG. 9 is an illustration of extraction of separate boundary contours for bone shaft, from an ML view x-ray image.
  • FIG. 10 is an illustration of template alignment with respect to Medial-Lateral image.
  • FIG. 11 is a flowchart of an example method of 3D image reconstruction from a single X-ray image.
  • FIG. 12A is a flowchart of an example method of 3D image reconstruction and template deformation separately with respect to ML and then AP x-ray image.
  • FIG. 12B is a flowchart of an example method of the 3D image reconstruction and template deformation simultaneously with respect to ML and then AP x-ray image.
  • FIG. 13 is a flowchart of an example method of determining alignment of the template with respect to the input x-ray image.
  • FIG. 14 is a flowchart of an example method of 3D image reconstruction from a two Orthogonal X-ray image.
  • FIG. 15 illustrates the imaging space with calibration parameters.
  • FIG. 16 depicts landmarks and axes on the bone.
  • FIG. 17 illustrates computation of landmark based on the standard directions of the bone’s anatomical co-ordinate system.
  • FIG. 18 illustrates computed anatomical landmarks.
  • FIG. 19 illustrates the 3D reconstruction flowchart.
  • FIG. 20 illustrates a flowchart for initial template alignment and condyle
  • FIG. 21 illustrates a flowchart for deforming a template bone.
  • FIG. 22 illustrates a flowchart for local deformation.
  • FIG. 23 shows the algorithm of deformity correction.
  • FIG. 25, FIG. 26, and FIG. 27 shows a typical deformity correction.
  • FIG. 28 illustrates calculated calibrated camera system in 3D.
  • FIG. 29 illustrates a flowchart explaining the correspondence in shaft deformation.
  • FIG. 30 shows anatomical coordinate system (ACS).
  • FIG. 31 illustrates a patient specific instrument.
  • FIG. 32 illustrates the patient specific instrument with a patient bone’s 3D model.
  • FIG. 33 illustrates an external-fixator holds the bone after deformity correction.
  • FIG. 34 illustrates a patient specific instrument, which is automatically designed, to make bone cuts and holes for fixator pins based on simulated external-fixator position of FIG. 30.
  • FIG. 35 illustrates three different types of pegs for a patient specific intrument (PSI).
  • PSI patient specific intrument
  • FIG. 36 illustrates a femur patient specific instrument (PSI).
  • FIG. 37 illustrates a tibia patient specific instrument (PSI).
  • PSI tibia patient specific instrument
  • FIG. 38 illustrates a mesh face visible along Anatomical Coordinate System
  • first, second, etc. may be used herein to describe various elements, these elements should not be limited to any order by these terms. These terms are used only to distinguish one element from another; where there are“second” or higher ordinals, there merely must be that many number of elements, without necessarily any difference or other relationship.
  • a first element could be termed a second element, and, similarly, a second element could be termed a first element, without departing from the scope of example embodiments or methods.
  • the term“and/or” includes all combinations of one or more of the associated listed items. The use of“etc.” is defined as“et cetera” and indicates the inclusion of all other elements belonging to the same group of the preceding items, in any“and/or” combination(s).
  • “3D” means 3 -dimensional
  • “2D” means 2-dimensional
  • the structures and operations discussed below may occur out of the order described and/or noted in the figures. For example, two operations and/or figures shown in succession may in fact be executed concurrently or may sometimes be executed in the reverse order, depending upon the functionality/acts involved. Similarly, individual operations within example methods described below may be executed repetitively, individually or sequentially, to provide looping or other series of operations aside from single operations described below. It should be presumed that any embodiment or method having features and functionality described below, in any workable combination, falls within the scope of example embodiments.
  • 3D virtual surgical planning may aid in determining the best plan and transferring it to reality.
  • surgery planning in a 3D view may be more accurate, realistic, and/or satisfying (to a surgeon as well as patient) as compared to a conventional process of 2D view-based planning.
  • 3D planning requires rendering of a 3D image from available data.
  • X-ray images may be used for 3D reconstruction so that computational devices like mobiles phones or tablet computers, which have relatively lesser computational prowess, can also be used for the reconstruction process.
  • Portability provided by such devices allows for greater flexibility in a healthcare environment.
  • Hard copies of X-ray images of the region of the patient's body for operation may not allow a surgeon to simulate post-operative conditions and/or may be an inconvenient way to perform measurements.
  • digital X-rays only provide 2D visualization of internal bone/joint anatomy and hence do not give accurate view, orientations, simulations, and/or feeling of surgery of a 3D environment.
  • a 3D surgical planning environment with 3D bone shapes may require a 3D virtual model of the bone. While such 3D models may be derived from CT scans of the bone anatomy of a patient, CT scans involve health risk, cost, and time, such that medical professionals may not prefer to perform surgery planning using CT scans. Moreover, 3D model reconstructions from CT scans are difficult on portable mobile devices, due to data size and computational requirements. Conversion of CT data to a 3D model is anyway time-consuming and requires significant manual inputs. Transferring CT scan data over the internet/network for various
  • the Inventors have newly recognized that conversion of 2D X-ray images into 3D models may solve the above and other problems. Converting 2D X-ray images into 3D models may be computationally heavy and/or require X-ray images to be input in a way requiring a radiologist or surgeon to take extra care and/or use a special imaging device or a calibration device.
  • 3D images/models of the bone can also be used for printing the bones into plastic models for informing patients about the surgery and/or training and real-model-based surgery planning. 3D models of bones can also be used for printing patient- specific instrumentation used in orthopedic surgeries. Use of 2D X-rays for 3D modelling does not require a patient to go under the health risk or expense of CT scanning. 2D imaging data is further much smaller and much more easily transferred than CT scan data for transfer to an instrumentation
  • the present invention is devices, software as stored or executed on tangible computer-readable media, and methods for converting 2D X-rays into full 3D pre- operation planning models.
  • the few example embodiments and example methods discussed below illustrate just a subset of the variety of different configurations that can be used as and/or in connection with the present invention.
  • the system and method of this invention relates to any elongate bone.
  • FIG. 1 is an illustration of a block diagram of an example embodiment system 1 useable to obtaining 3D images using conventional 2D X-ray images.
  • 3D models of bones may be generated from one or two 2D X-ray
  • Example embodiment system 1 is processor-based, and actions of system 1— and where example embodiment system 1 executes example methods— are dependent upon the processor(s) being specially-configured for the same.
  • an X-ray inputter 12 provides X-ray images for conversion. Inputter 12 may acquire the X-ray images through known procedures with conventional single- view X-ray imaging equipment. Orthogonal X-ray images from biplanar imaging may also be used. Such X-ray images from inputter 12 may include medial-lateral and anterior-posterior views. The X-ray images may not have any markers and/or have any known orientation with respect to the bone.
  • a data importer 14 may import a patient's X-ray image(s) in digital format.
  • importer 14 may be a scanner configured to convert X-rays in hard copy format to a digitized format. This digitization may be done simply by using a camera, an X-ray digitizer, and/or an X-ray film scanner that converts the X-rays into digital format, such as any of the formats selected from JPG/TIF/PNG or DICOM format and the like.
  • the X-ray images imported can belong to medial-lateral (ML) view or anterior-posterior (AP) view or both. Such imported images, may be processed for 3D reconstruction as final X-ray images in a digital format.
  • a camera model determinator 18 b may detect whether an X-ray image is ML or AP, using known parameters.
  • image plane 101 is a plane in a 3D imaging space that corresponds to detector plane 101, a plane coinciding with the flat X-ray sensor panel or a film of the real imaging environment, where the projection of the body/object/bone is formed.
  • Image center 102 is the central position of a rectangular detector.
  • image center 102 may be the normal position on image plane 101, which coincides with the X-ray source, such as an X-ray sensor panel or a film is as placed during the imaging.
  • the determined camera model is used for 3D reconstruction to mimic the real X- ray imaging environment and includes the following: position of X-ray source 104, such as a point source corresponding to real X-ray source of the imaging equipment, with respect to image plane 101 in the imaging space; and the distance 103 between centroid 106 of an object such as bone 50 and the X-ray source 104, measured in the direction normal 107 to image plane 101 in the imaging space.
  • source film distance (SFD) 105 source object distance (SOD)
  • Position of the X-ray source 104 with respect to image center 102 is determined so that a normal of image plane 101 arising from image center 102 will coincide with source 104 at a known distance called source film distance 105 from image center 102.
  • SFD 105 is equal to the distance between an X-ray source 104 and the detector, measured along the direction that is normal 107 to detector plane 101.
  • Source object distance 103 may be defined as the distance between X-ray source
  • a camera calibration perspective ratio K may be defined as a ratio of SOD 103 to SFD 105.
  • SOD 103 may either be a known parameter or may be approximated. An example method to determine SOD 103 approximately is disclosed as below.
  • a spherical ball marker with a known actual diameter (for example, 25 mm) is placed near the object (bone 50/body) during X-ray imaging, closer to image center 102, at a height from detector plane 101, that is closer to the height of centroid 106 from detector plane 101, by eyeballing.
  • SOD 103 will be equal to multiplication of SFD 105 and the ratio of the known actual diameter of the spherical ball marker to the diameter of the circular/elliptical projection of the spherical ball marker on detector plane 101.
  • the diameter of the circular/elliptical projection of the spherical ball marker on detector plane 101 is equal to the diameter of the circular/elliptical projection of the spherical ball marker measured on the final X- ray image multiplied by the digital magnification ratio (given below).
  • a digital magnification ratio determinator for an X-ray image may be included in example embodiments.
  • the digital magnification ratio is the ratio of the value of the distance between the positions of the projections of any two points on the object's surface on detector plane 101 to the value of the distance between the corresponding points as measured in the final X-ray image, which may be measured in terms of pixels or mm.
  • This ratio can be a known parameter, or an example method for determining the digital magnification ratio for an X-ray image may be used wherein a circular coin marker with known actual diameter is placed on the detector while taking the X-ray image.
  • the digital magnification ratio will be approximately equal to the ratio of the known actual diameter of the circular coin to diameter of the coin as visible on the final X-ray image, as measured in terms of number of pixels or mm. All the positions determined on the final X-ray image, in terms of X and Y coordinates (e.g., in pixels) may be multiplied with the digital magnification ratio before processing for 3D reconstruction. This includes contour points and landmarks.
  • example embodiment system 1 may include a contourer 16 that defines contours of a bone or other object in an uploaded or imported X-ray.
  • the contour of bone is a curve consisting of set of 2D points on the final X-ray image which corresponds to the outer boundary of the bone that is visible on the final X-ray image.
  • Contourer 16 may allow a user to draw an outer boundary of the bone anatomy of interest.
  • a user draws the outer boundary of the bone anatomy of interest, depending on the surgery.
  • a femur and tibia bone for knee replacement or tibial osteotomy surgery may be outlined.
  • Automated pre- defined contouring may be used to pre-empt contouring lines and assist the user in relatively more precise contouring. Brightness and/or contrast of the X-ray image may be adjusted so that the boundary of bone anatomy is easily distinguishable.
  • Contourer 16 may provide an initial contour for each bone that can be boundary of the projection of the template according to the calculated camera model. Since the vertices of the template will be divided and labelled as distinct regions, the projected initial contour will also have the distinction of the predetermined regions. A user may modify the initial contour to fit the bone's outer edge or boundary more precisely; the modification entails scaling, translation, rotation, deformation, etc. Contourer 16 may provide a touch interface wherein a user can touch a bone's boundary on the X-ray image and the contouring mechanism converts the touch interfaces to points, lines, and provides a continuous pattern in an intelligent manner.
  • contourer 16 Defining contours using the contourer 16 is provided to define co- ordinates of the contour of the bone with respect to a relative or pre-defined center of an X-ray image.
  • the X-ray in the medial-lateral plane is the x-z plane for the purposes of this invention.
  • the X-ray in the anterior-posterior plane is the y-z plane for the purposes of this invention.
  • Anatomical regions may give anatomical landmarks to define anatomical parameters.
  • Anatomical landmarks may be used for alignment of templates, and anatomical parameters may be used for selective anatomical modification of pre- created 3D templates.
  • a 2D Anatomical Value may include: anatomical
  • landmarks 2D positions of unique anatomical features identified on the final X- ray image on the basis anatomical regions
  • anatomical parameters values of geometric parameters like lengths and angles calculated based on anatomical landmarks to be used for 3D reconstruction.
  • the points of the contour of bone may be divided into subsets in such a way that the subset points correspond to distinct anatomical regions of the bone.
  • FIG. 3A shows the anatomical regions.
  • a contour of the bone in at least one view (ML or AP) of an X-ray image the anatomical regions will be: femoral lateral condyle; femoral medial condyle; femoral shaft; femoral neck; femoral trochanter; and femoral ball.
  • a contour of the bone in at least one view (ML or AP) of X-ray image the first view (ML or AP) of an X-ray image.
  • anatomical regions will be tibial proximal condyle, tibial shaft, and tibial distal condyle.
  • the anatomical regions may be distinguished by drawing different regions of the contour in different colors if the contour is determined by drawing manually.
  • anatomical axes are also determined manually or automatically.
  • the anatomical axis, shaft axis, and the neck axis may be determined.
  • the anatomical axis and shaft axis may be determined.
  • a line may be fitted along user specified points that lie on the axis in the image.
  • a user may place a given line or curve (in case of shaft axis) along the position and orientation of the required axis.
  • a geometric calculation is performed on the distinguished anatomical regions of the contour. For example, a best fit line to the femoral shaft region of the contour may be assigned as the femoral anatomical axis. Or, for example, a best fit Bezier curve to the femoral shaft region of the contour may be assigned as the femoral shaft axis. Or, for example, a best fit line to the femoral neck region of the contour may be assigned as the femoral neck axis.
  • a best fit line to the tibial shaft region of the contour may be assigned as the tibial anatomical axis.
  • Positions of anatomical landmarks may be determined on the final X-ray image with respect to the extracted contours, based on anatomical regions.
  • FIG. 3B shows the anatomical landmarks and the anatomical parameters.
  • a user may specify points on the image that lie on the landmark.
  • an automatic method of determination of anatomical landmarks a user may specify points on the image that lie on the landmark.
  • the anatomical landmarks may be determined from the final X-ray image by calculating geometric features, such as extreme position in a direction, or a centroid, or a peak, of the above-mentioned anatomical regions of the contour maybe with respect to some known anatomical axes.
  • Femoral Distal-Lateral condylar landmark a position of the extreme distal point along the Femoral anatomical axis of the Femoral lateral condyle region of the contour
  • Femoral Distal-Medial condylar landmark a position of the extreme distal point along the Femoral anatomical axis, of the Femoral medial condyle region of the contour
  • Femoral landmark a position of the extreme lateral point along the line passing through the Femoral Distal-Lateral condylar landmark and the Femoral Distal-Medial condylar landmark
  • Femoral Medial condylar landmark a position of the extreme medial point along the line passing through the Femoral Distal-Lateral condylar landmark and Femoral Distal-Medial condylar landmark
  • Femoral ball landmark an average of the position of the center of the best fit sphere to all the points of the femoral ball region of the contour
  • Greater Trochanter tip landmark a position of the extreme proximal point of the Femoral trochanter region of the contour
  • Shaft-Neck landmark a position of the intersection of the femoral anatomical axis and the AP femoral neck axis.
  • Femoral Distal-Lateral condylar landmark a position of the extreme distal point along the Femoral anatomical axis, of the Femoral lateral condyle region of the contour
  • Femoral Distal-Medial condylar landmark a position of the extreme distal point along the Femoral anatomical axis, of the Femoral medial condyle region of the contour
  • Femoral Posterior-Lateral condylar landmark a position of the extreme posterior point perpendicular to the direction of femoral anatomical axis, of the Femoral lateral condyle region of the contour
  • Femoral Posterior- Medial condylar landmark a position of the extreme posterior point perpendicular to the direction of femoral anatomical axis of the Femoral medial condyle region of the contour
  • Femoral Anterior-Lateral condylar landmark a position of the extreme distal point along the Femoral anatomical axis, of the Femoral
  • Greater Trochanter tip landmark a position of the extreme proximal point of the Femoral trochanter region of the contour.
  • Tibial Proximal-Lateral condylar landmark a position of the Extreme lateral point perpendicular to the direction of tibial anatomical axis, of the tibial proximal condyle region of the contour
  • Tibial Proximal-Medial condylar landmark a position of the Extreme medial point perpendicular to the direction of tibial anatomical axis, of the tibial proximal condyle region of the contour
  • Tibial Distal- Lateral condylar landmark position of the Extreme lateral point perpendicular to the direction of tibial anatomical axis, of the tibial distal condyle region of the contour
  • Tibial Distal-Medial condylar landmark position of the Extreme medial point perpendicular to the direction of tibial anatomical
  • Tibial Proximal-Posterior condylar landmark a position of the Extreme posterior point perpendicular to the direction of tibial anatomical axis, of the tibial proximal condyle region of the contour
  • Tibial Proximal- Anterior condylar landmark a position of the Extreme anterior point perpendicular to the direction of tibial anatomical axis, of the tibial proximal condyle region of the contour
  • Tibial Distal- Posterior condylar landmark a position of the Extreme posterior point
  • Tibial Distal- Anterior condylar landmark a position of the Extreme anterior point perpendicular to the direction of tibial anatomical axis of the tibial distal condyle region of the contour.
  • Anatomical Parameters may be calculated automatically based on anatomical landmarks; parameters can be a distance between two landmarks, an angle between lines defined by any two landmarks, and/or any correlative value between landmarks.
  • Femoral Medial-Lateral condylar width the distance between femoral Lateral condylar landmark and femoral Medial condylar landmark
  • Femoral Shaft length the distance between femoral shaft-neck landmark and a position of intersection of femoral AP anatomical axis and a line connecting Femoral Distal- Lateral condylar landmark and Femoral Distal-Medial condylar landmark
  • Length of Femoral Mechanical axis the distance between femoral ball landmark and the center of Femoral Distal-Lateral condylar landmark and Femoral Distal-Medial condylar landmark
  • Femoral Neck length the distance between AP Femoral
  • Tibial Medial-Lateral condylar width the distance between tibial Proximal-Lateral condylar landmark and tibial Proximal-Medial condylar landmark
  • Tibial Shaft length the distance between a position of intersection of tibial AP anatomical axis and a line connecting tibial Proximal-Lateral condylar landmark and tibial
  • bone template model inputter 18 a may provide a corresponding bone template model in 3-dimensional format.
  • the corresponding bone template model format may be a clinically normal bone in the form of 3D mesh with triangular elements.
  • This bone template model may be reconfigured into a shape that matches the input contours as defined by contourer 16.
  • the pre-created 3D template may be formed in the form of mesh, pre-created from a CT scan of some healthy/average subject or subject with matching medical condition to a patient whose input X-ray images are used for the 3D reconstruction.
  • a data set with multiple subjects may be created. Demographics and gender of subjects may be used to make discreet the data set.
  • Different template shapes belonging to different ages or age groups, ethnicity groups, etc. may be created and stored.
  • a 3D surface model can be created using techniques such as MIMICs through segmentation of all the slices images of CT scan.
  • the surface model can be exported as point cloud surface model.
  • a point cloud is a set of data points in some coordinate system. In a 3D coordinate system, these points are usually defined by X, Y, and Z coordinates and are often intended to represent the external surface of an object (such as bone 50).
  • Connectivity between points of the point cloud can be formed using methods like constrained Delaunay Triangulation to form a 3D mesh model with triangular elements.
  • a triangular element is an element which is defined by forming connectivity between three points. By triangulation of all the points of the point cloud a mesh of triangular element may be formed.
  • the point cloud may be sampled to reduce the number of surface points, and hence the number of triangular elements resulting from meshing.
  • sampling related parameters such as reduction in volume formed by the closed mesh, may be defined to form an optimum model such that errors are minimum and bone shape features are preserved, but points are relatively reduced.
  • a surface model may be exported from a dense cloud— for example, a cloud with 1 mm point-to-point mesh distance.
  • the surface model may then be uniformly sampled to a sufficient number of points.
  • a sufficient number of points may be determined by measuring the level of detail of the 3D bone model.
  • the level of detail and the volume (of the closed meshed model) gets reduced after the sampling.
  • the reduction in level of detail can be determined by measuring the difference in volume of a closed mesh created from the initial dense point cloud and that of a closed mesh created from the sampled points. By putting the threshold on the level of detail, such as a volume reduction of 2%, the sampling and sufficient number of points may be determined.
  • the point-to-point distance at this condition in an example of a femur bone template, may be 3 mm.
  • a 3D mesh with triangular elements may be created from the sampled points and used as the template model for the 3D reconstruction.
  • the template model may be in the form of triangular surface mesh with sets of a number of vertices and a number of faces.
  • the number of vertices may be 1795 and the number of faces may be 3559, for example.
  • the anatomical regions, anatomical axes, anatomical landmarks, and anatomical parameters of the 3D template model may be pre-determined, at least manually.
  • an improved 3D reconstruction system and method for deformed elongate bones.
  • This system and method requires four input items: i) Calibration parameters obtained from a calibrator 18/;
  • Contour points obtained from the con tourer 16, from 2D X-ray images taken in approximate AP and approximate ML view;
  • Landmark points obtained using camera model determinator 16, from 2D X-ray images taken in approximate AP and approximate ML view;
  • Calibration parameters include position of X-ray source: S(x, y, z), Source Film distance: SFD, principal point position: PPos (X, Y), vectors dirX and dirY representing orientation of the image plane in 3D space.
  • the calibration parameters for AP and ML images are in the same 3D space since a single calibration marker object is used while taking both images. This 3D imaging space is transformed such that the vectors dirX and -dirY for AP image align with XZ plane. Using the calibration parameters, the X-ray imaging can be simulated for 3D reconstruction.
  • FIG. 15 shows the imaging space with calibration parameters.
  • Contour points Cn (X, Y) and landmark points Lk (X, Y), for a bone are in 2D coordinate system as they are extracted, by a user, from a 2D X-ray image.
  • the landmark points Lk include all essential landmarks of the bone to represent its deformity.
  • Table 1 lists down some example anatomical landmarks.
  • FIG. 16 depicts the anatomical landmarks and anatomical axes on a bone.
  • Table 1 Anatomical Landmarks and Anatomical Axes (Joint line, mechanical) extracted from lower limb AP and ML X-ray images
  • FIG. 17 illustrates computation of landmark based on the standard directions of a bone’s anatomical co-ordinate system.
  • the contour points Cn are the boundary of those selected regions of lower limb bones which represents torsional deformity and knee joint deformity.
  • the regions include distal femur and proximal tibia in both AP and ML X-ray image.
  • the distal femur contour is divided into 3 parts i.e. condylar shaft, medial condyle, and lateral condyle.
  • the proximal tibial contour is a single condyle part.
  • the template is a 3D model of a bone in the form of mesh (object with vertices and faces) with triangular elements.
  • the vertices of this mesh represent the points on the surface of the bone (femur or tibia).
  • This template is modified through various steps into a final required 3D bone model.
  • the template segmentation process is defined by at least the following steps:
  • Step 1 Calculating the three principal axes by applying principal component analysis (PCA) on the vertices 3D position data of the template.
  • PCA principal component analysis
  • Step 2 Rotating the template in such a way that first principal axes matches with Z axis, second principal axes with X axis, and third principal axes with Y axis respectively.
  • the standard coordinate axis can be considered as an initial estimate of ACS for the template bone.
  • Step 3 Separating the middle 3/5 th vertices of the template bone along Z axis and finding the best-fit cylindrical axis to these vertices region.
  • Step 4 Realigning the template such that the best-fit cylindrical axis becomes parallel to Z axis
  • Step 5 Extracting the bottom l/5th vertices and top l/5th vertices of the template along Z axis as distal and proximal segment (storing indices) respectively.
  • the rest of the vertices are shaft segment.
  • Step 6 The distal condyle segment is divided into three sub-segments namely medial condyle, lateral condyle and condylar shaft. The top half is condylar shaft while bottom half is divided further into right and left half as medial and lateral condyle respectively (for right bone template).
  • Step 7 The proximal condyle segment is divided into four sub-segments namely femoral ball, femoral neck, femoral greater trochanter and femoral lesser trochanter. From the proximal segment, the extreme points along positive Z axis and positive X axis is calculated and an initial estimate of femoral ball sphere radius and centre is calculated (FIG. 17).
  • Vertices included in a spherical region with 1.1 times the estimated radius around the estimated centre are extracted. Best- fit sphere radius and centre is determined for the extracted region.
  • the vertices with the coefficient of determination (R2) below a threshold are extracted as the femoral ball segment.
  • the threshold is the value at which the R2 suddenly jumps.
  • Step 8 The remaining region tangent to sphere is extracted as neck, greater trochanter and lesser trochanter
  • FIG. 3C illustrates these anatomical regions as corresponding to the regions distinguished in the contour of an X-ray image.
  • Anatomical landmarks identified based on anatomical regions of the template may be the same as the anatomical landmarks identified based on anatomical regions of the contour, as shown in FIG. 3D.
  • Anatomical parameters identified based on anatomical landmarks of the template may be the same as the anatomical parameters identified based on anatomical landmarks of the contour.
  • landmarks are calculated based on the standard directions of bone’s ACS. In the given ACS (initially estimated above), the following 3D landmarks are calculated.
  • the anatomical parameters defining the deformity in a bone are calculated along standard directions like Anterior-Posterior (AP) and Medial-Lateral (ML) and Superior-Inferior (SI).
  • AP Anterior-Posterior
  • ML Medial-Lateral
  • SI Superior-Inferior
  • torsional deformity is measured as angle between femoral ball neck axis and posterior condylar axis, along the SI direction.
  • a condylar deformity is measured as an angle between mechanical axis and distal joint line along both AP and ML direction separately.
  • These three directions constitute the ACS for a bone and can be calculated based on the anatomical landmarks.
  • the Z-axis (SI direction) of the ACS is along the mechanical axis of the bone.
  • the Y-axis (AP direction) of the ACS is along the cross-product of the mechanical axis and the posterior-condylar axis.
  • the X-axis (ML direction) of the ACS is the cross-product of the Y-axis and the Z-axis of the ACS.
  • the landmarks can be re-calculated based the new ACS and vice-versa iteratively.
  • FIG. 18 illustrates computed anatomical landmarks.
  • FIG. 19 illustrates the 3D reconstruction flowchart.
  • contour and landmark points in 2D coordinate system are transformed into 3D coordinate system (Cn (x, y, z) and Lk (x, y, z)) of the imaging space. This is done using the calibration parameters for both AP and ML X-ray images. This is followed by the alignment of the template in the 3D imaging space. The template is then deformed at various regions to reconstruct the deformity in input bone. This includes shaft bending deformity, condyle region deformity and torsional deformity. Finally, the bone template is deformed in such a way that its projection on the X-ray image plane in the imaging space will match exactly with the input contour.
  • 3D coordinate system Cn (x, y, z) and Lk (x, y, z)
  • Principal point PPos(x, y, z) is defined as a 3D point at the distance SFD from the source in the direction normal to dirX and dirY.
  • the transformation parameters are calculated in such a way that PPos(X, Y) transforms to PPos(x, y, z), X and Y coordinate axis of the image coordinate system aligns with dirX and -dirY vector respectively.
  • Cn(x, y, z) and Lk(x, y, z) of AP and ML images are in same 3D space because respective calibration parameters are also in same 3D space.
  • FIG. 20 illustrates a flowchart for initial template alignment and condyle deformation.
  • the second step is performed using ICP (Iterative closest point) based method.
  • ICP Intelligent closest point
  • 3D point-pairs required for the ICP are calculated separately for medial condyle, lateral condyle and condylar shaft. This results in better 3D alignment compared to when the 3D point-pairs are calculated for the whole condyle altogether (especially in the ML view). This is because the separate point-pairs result in the accurate relative positioning of medial and lateral condyle which in-tum results in accurate alignment along the shaft axis.
  • This process is required to reconstruct the bending and torsional deformity in the bone.
  • the shaft axis landmarks in AP and ML images have end to end
  • FIG. 21 illustrates a flowchart for deforming a template bone
  • the template shaft deformation includes calculation of template shaft axis and deforming the shaft segment such the template shaft axis matches with the reference shaft axis.
  • the template shaft segment is divided into 10 sub- segments along its first principal axis.
  • the centroids of vertices belonging to the open boundary of shaft segment mesh are calculated at its distal and proximal ends (boundary-centroids).
  • the template shaft axis is calculated as a Bezier curve with m number of uniform points passing through centroid of each sub- segment and the boundary-centroids.
  • the shaft segment vertices are re-divided into m-1 number of sub-segments based on their positions with respect to the m-1 line segments of template shaft axis.
  • Affine transformations are calculated for each m-1 line segments of the template shaft axis so that they coincide with corresponding m-1 line segments of the reference shaft axis. The same transformations are applied to the associated shaft sub- segments.
  • the m number should be sufficiently large to get a smoothly deformed (bending deformity) shaft segment.
  • a reference femoral ball centre is calculated.
  • the position of the reference femoral ball centre with respect to the reference shaft axis represents the torsion in the bone.
  • the template shaft sub-segments are twisted. This is done in a distributed way by appropriate rotation of each m- 1 shaft sub-segments about their associated line segments of the template shaft axis. This brings the accurate torsion in the shaft but the template femoral ball centre will not exactly match the reference femoral ball centre.
  • FIG. 22 illustrates a flowchart for local deformation.
  • the local deformation results in an accurate surface reconstruction of the bone.
  • the local deformation may be performed using Laplacian surface deformation.
  • Laplacian surface deformation smoothly deforms a mesh while bringing a few selected anchor points of the mesh to respective target positions (positional constraints) and maintaining the inter- vertices positional relationship (Laplacian constraints) described by Laplacian coordinates.
  • a least square method is applied to follow both positional and laplacian constraints.
  • the anchor points are silhouette points of the template bone for AP and ML view calculated using respective X-ray source positions and image planes.
  • the silhouette points are projected on the image planes and their corresponding contour points are identified based on Self organizing maps as explained in et.al. From each corresponding contour point, a ray is back projected to the source and a nearest position on the ray from each silhouette point is determined. This nearest position is the target position for the silhouette point (anchor).
  • a 2D-to-3D converter 18 converts the 2D X-ray images to 3D images.
  • the conversion may be based on Laplacian deformation, which is an efficient shape deformation technique.
  • the generated 3-dimensional model may a surface model and/or a solid model, with the surface model having reduced computational requirements.
  • a silhouette vertices extractor 18 d in converter 19 may extract silhouette vertices and projections of a 3-dimensional template, at its aligned position, in accordance with the determined camera model, using known parameters.
  • Silhouette vertices are those vertices of the template which form the outer contour of the template's projection on image plane 101, according to camera model, hereinafter called a template projection contour.
  • a perspective projection of the vertices of the template mesh may be computed on its image plane.
  • the outer contour of the template projection, or template projection contour can be computed using the following example method. All vertices of the template may be projected on image plane 101
  • Triangulation meshing of projection is obtained by using Delaunay triangulation method (2DM).
  • 2DM Delaunay triangulation method
  • a 2D mesh (2CDM) with triangular elements is created from the projected points as seen in FIG. 4, illustrating triangulation of projected points, meshing after putting constraints and the outer contour calculation.
  • Those edges of the triangular elements which are shared with only one triangular element are the boundary edges and the corresponding projected points are the boundary point and hence the template projection contour points.
  • the silhouette vertices are those vertices of the template which form the outer contour of the template's projection (template projection contour) on image plane 101, according to a camera model.
  • An example embodiment 2D-to-3D converter 18 may include an aligner 18 c that aligns a pre-created 3-dimensional template of a bone with respect to the contour points.
  • the pre-created 3 -dimensional template may be formed in a mesh, pre- created from a CT scan of some clinically normal bone, such as from a data set with multiple subjects. Alignment of the pre-created 3 -dimensional template differs according to the image view and bone anatomy. For example, the image view may be one from medial-lateral or one from anterior-posterior.
  • Alignment may be performed in the context of a femur bone, for example.
  • Converter 18 may include anterior-posterior pose estimator 22 configured to determine a first alignment of a femoral template with respect to the anterior- posterior input X-ray image.
  • Input to estimator 22 may be taken from the contourer 16, which has contoured data and image of a bone's X-ray in its anterior-posterior view.
  • a joint center may be located, and the template projected on to an image plane with arbitrary initial positions and orientation. This assists in deformation of the femoral template for 3D reconstruction.
  • the template models (femur and patella), obtained from the bone template model inputter 12 may be in the form of surface point cloud.
  • a source-film distance 105 is calculated, and a source-object distance 103 is calculated.
  • the projection may be determined as perspective type and calculated according to a camera model.
  • an automatic initialization may place the contour points on image plane 101 of the camera model.
  • the template may be positioned and/or translated between X-ray source 104 and image plane 101 of the camera model, in such a way that the template's centroid 106 is at the distance of SOD 103 from the X-ray source 104, measured along a normal 107 to image plane 101. Centroid 106 may be defined as the average of the positions (x,y,z) of the vertices of the template. Orientation of the template may make image plane 101 parallel to that plane of the template (ML or AP) of which the contour belongs to.
  • the template may be rotated about the normal to image plane 101 passing through the template's centroid 106, in such a way that the projection of its anatomical axis (by the camera model) becomes parallel with the anatomical axis of the contour.
  • the templates may be translated along directions parallel to image plane 101 in such a way that centroid 106 of the bone template projection coincides with that of the contour.
  • a two-step procedure may be applied to find the template's pose in 3D.
  • a patellar template may be rigidly translated or rotated with the femoral template.
  • the templates femur and patella
  • the templates are rotated about an anatomical axis, e.g., parallel to Z-axis, to match the position of the joint center with respect to the template in its projection on image plane 101 with that in the input contour.
  • the horizontal distance, measured along a direction perpendicular to anatomical axis and a normal to image plane, “dcml” between the joint center and the anatomical axis is calculated from the input contour.
  • the ratio“rcml” of distance“dcml” to medial-lateral width “dcml”— distance between femoral Lateral condylar peak and femoral Medial condylar peak— of the femur bone is also calculated from the input contour.
  • an angle of rotation about the anatomical axis can be calculated using the relation between the distance“dcml” and patellar angle as shown in FIG. 9. After rotation about the anatomical axis, distance, and hence ratio, changes. Hence, the process is applied iteratively until the difference rpml- rcml becomes very small.
  • the joint center is the position of the centroid of the points of the contour of patella bone visible on the X-ray image.
  • the joint center is the position of the centroid of the points of projection of the template of Patella bone, which is always rigidly positioned with respect to the femur bone template.
  • the input contour and the template projection are first processed for the equivalence in shapes.
  • the input contour of the bone was truncated to match its aspect ratio to that of the projection.
  • the outer boundary of the femoral template projection is extracted automatically using the silhouette vertices' extraction.
  • Step 2 the extracted femoral template projection contour is aligned to the input contour using a shape registration method like iterative closet point analysis (ICP).
  • ICP iterative closet point analysis
  • Optimal values transformations are calculated using ICP, for the template projection contour to align it with the input contour.
  • Corresponding transformations are applied to the template in such a way that its projection on image plane 101 (after applying transformations) will match with the aligned template projection contour.
  • 3D- 3D point pairs are determined after the final alignment of template projection with the contour points of anterior-posterior view. This may be performed using a back projection method.
  • Input contour 201 is provided by a user using an X-ray image.
  • template projection contour 202 that is input using the system and method of this invention, which template projection contour is provided before alignment.
  • Aligned template projection contour 203 may be provided after alignment of the template projection with respect to the input contour defined by the user.
  • a silhouette vertex of the template with its initial position as m 204 corresponding to the template projection contour point pp m b 205, a closest position bs m 206 on the projection ray r m 207 joining the X-ray point source 104 and the corresponding aligned template projection point pp m 208 is calculated using a template projection point pp m b 205 available before alignment.
  • total M numbers of 3D-3D point pairs (as m , bs m ) are found for each silhouette vertex. ICP technique was applied on these point pairs (as m 204, bs m 206) to find the transformations of silhouette vertices 301 for their optimal
  • FIG. 6 shows the template model before and after the alignment.
  • a new corresponding points' pair between template projection and input contour may be determined.
  • the mean absolute distance (MAD) between the points of template projection contour and their corresponding closest points of the input contour may be measured.
  • the iteration is stopped when the difference in MAD of the two consecutive steps of iterations is below 0.0001 mm.
  • the MAD between the input contour and the template projection contour is minimized through the iteration.
  • the corresponding alignment of the 3D template is then applied at once.
  • Example embodiment system 1 may include a medial-lateral pose estimator 24 configured to determine a second alignment of the template with respect to the input X-ray image, for a femur bone shape.
  • Input to estimator 24 may be taken from contourer 16 which has contoured data and image of a bone's X-ray in its anterior-posterior view.
  • An anterior-posterior projector projects the anterior- posterior image on to an image plane with arbitrary initial positions and
  • the template model of femur obtained from the bone template model input mechanism, is in the form of surface point cloud.
  • FIGS. 9 and 10 from the ML view X-ray image, separate boundary contours may be manually extracted for bone shaft, medial bone side, and lateral bone side.
  • FIG. 9 illustrates template alignment with respect to Anterior-Posterior image
  • FIG. 10 illustrates template alignment with respect to Medial-Lateral image.
  • the automatic initialization process may be similar as that for the anterior- posterior view. After the initialization, the two-step procedure is applied.
  • the template is first rotated about the shaft axis. For this, a ratio“rcapd” of distance between Posterior-Lateral condylar peak and Posterior-Medial condylar peak of the bone to the anterior-posterior width, both measured along direction perpendicular to anatomical axis and a normal to image plane, may be calculated from the contour in FIG. 10. Similar ratio“rpapd” may be calculated from the template projection on image plane.
  • the template is rotated about the anatomical axis so that the ratio“rpapd” matches with the ratio“rcapd.” The angle of rotation may be calculated using a trigonometric function.
  • the template is then rotated about an axis that is direction perpendicular to anatomical axis and a normal to image plane and passing through its centroid.
  • a ratio“rcapp” of distance between Femoral Distal- Medial condylar landmark and Femoral Distal-Lateral condylar landmark is calculated.
  • step 1 is applied to find optimum translation, rotation, and scaling using a shape registration method like ICP, in the same way as it is applied for the anterior-posterior view. If the two images are exactly orthogonal to each other from bi-planar X-ray imaging, refer to FIG. 14.
  • the template may be aligned in 3D space to match its projection contours, i.e., the template projection contours, with respect to both AP and ML contours simultaneously, using a shape registration method like ICP.
  • Optimal values transformations may be calculated using ICP, for the template to align it with both the input contours (ML and AP).
  • the camera model with respect to the ML and AP view X-ray image are combined.
  • the ML and AP view image planes and image centers have known fixed relative position and known fixed relative orientation (usually 90 degree) with respect to each other.
  • the two camera models (for ML and AP view) are combined in one imaging space and include, two X-ray point sources, two image planes orthogonal to each other, and known SFD (source-film distance).
  • a position of template is found in the imaging space in such a way the template projection contours on both image planes (calculated according to corresponding camera models) aligned with the shape of the corresponding contours.
  • the template is rotated and translated in the imaging space and the optimal rotation and translation parameters are found using modified ICP based method.
  • Example embodiment system 1 may include a selective anatomical modifier 26 for global matching configured to selectively modify anatomical regions by scaling, translation, and/or rotation to match the 2D projections of its anatomical
  • landmarks, axes, and parameters with the 2D anatomical parameters extracted from the final X-ray image may be done with respect to the ML and AP image for a truncated distal femur or proximal tibia.
  • the corresponding template may be uniformly scaled along all three directions (X, Y, and Z) to match the medial-lateral width of distal femoral condyle or proximal tibial condyle approximately.
  • additional steps may be performed to match the shaft length, shaft axis and neck axis.
  • the template's shaft part region may be scaled along the anatomical axis to match the length of 2D projection of the anatomical axis with the corresponding length in the input X-ray image.
  • the femoral shaft region may be divided into sub-regions along the shaft- axis.
  • the femoral shaft region may be sheared where sub-regions may be translated, bent where sub-regions may be rotated, and/or twisted where sub-regions may be rotated along shaft axis in such a way that the 2D projection of its shaft axis matches with the shaft axis in the input X-ray image.
  • the femoral trochanter, neck, and ball regions may be sheared, scaled, bent, twisted, translated, and rotated along its neck axis to match the positions of the Femoral ball landmark, the Femoral greater trochanter tip landmark in the input X- ray image with the 2D projections of the corresponding landmarks of the template.
  • the shaft length may be matched by scaling the template's shaft part along its anatomical axis to match the length of 2D projection of the anatomical axis with the corresponding length in the input X-ray image. All these operations may be performed while preserving connectivity between parts (neck, ball, shaft etc.).
  • 2D values of the anatomical parameters of extracted from both AP and ML images may then be combined according to the determined camera model to get their 3D values with a 3D geometric calculation mechanism (standard 3D geometry method).
  • the template is then selectively modified where regions or sub-regions may undergo transformations like scaling, shearing, translation, and rotation to match the 3D value of its landmarks, axes and anatomical parameters with the 3D values of the anatomical parameters calculated from the 2D values extracted from the AP and ML images.
  • Example embodiment system 1 may include a template deformer 18 e configured to deform a standard template model in accordance with defined contours and silhouette vertices obtained from the bi-planar X-ray images.
  • Deformation may include deforming the transformed template mesh in such a way that the silhouette vertices get their target position (which will be determined using a SOM technique explained below) while preserving the overall topology and differential property of the transformed template.
  • FIG. 8 illustrates deformation using Laplacian surface deformation (LSD).
  • Each vertex of a mesh 401 is represented as a differential coordinate, which is the difference between the position of vertex and that of its neighbor vertices 402.
  • the inputs are the initial mesh, a set of anchor points (a few vertices of the initial mesh) and target positions of the anchor points.
  • the output is a deformed mesh where the anchor points take the target positions while preserving the local shape features and topology of the initial mesh.
  • the template mesh model may be input as the initial mesh, the silhouette vertices with initial positions 403 are the anchor points, and the target positions 404 of the silhouette vertices are the target positions of the anchor points.
  • the differential coordinate 405 for each vertex 401 is defined as the vector from the coordinates of the centroid of its immediate neighbors to its coordinates.
  • the template deformation may be performed using a Laplacian Surface
  • the template projection contour points may be adapted to the input contour using a self-organizing maps (SOM) technique.
  • the top contour is template projection contour 202.
  • Black contour is the input contour.
  • the lower contour is the adapted template projection contour 501 obtained by deforming template projection contour 202 using a SOM technique. This is how to find 2D-2D correspondence.
  • SOM technique smoothly deforms the projection contour and preserves the topology (connectivity).
  • the nearest point of the projection contour may be identified and partially pushed toward the contour point.
  • the neighboring points of that particular projection point may also be pushed toward the input contour point.
  • their motion is controlled by a specific neighborhood which is an exponential function whose value is high for the projection contour points that are closer to the winner and small for points which are farther away.
  • the adaptation process lessens smoothly with time and controlled by another exponential function called learning rate.
  • SOM gives the 2D-2D correspondence— template projection contour points— adapted template projection contour points between template projection contour 202 and adapted template projection contour 501.
  • 3D-3D correspondence point pairs may be calculated for the silhouette vertices by the back projection method of FIG. 5.
  • the adapted template projection points were back projected to find target positions of corresponding silhouette vertices.
  • the silhouette vertices— their target positions— may be the 3D-3D point pairs.
  • the 3D-3D point pairs may be used as positional constraints for LSD.
  • the inputs of the LSD were the template mesh, the silhouette points which will act as the anchor points, and target positions of the silhouette points which were included in the 3D-3D point pairs.
  • Each vertex of the mesh is represented by the differential coordinate that is a difference between the position of a vertex and the centroid of the neighboring vertices in the mesh.
  • the anchor points are forced towards their targets while preserving the differential property of the mesh vertices, causing smooth deformation with preservation of shape features.
  • a matching point analysis may compute and provide at least a best matching point, for each of the template projection contour point(s) that correspond to the silhouette vertex position(s), on the input contour of the bone, such as 2D-2D correspondence using the SOM method.
  • Deformation may further include constructing a correspondence map for converting points from the 2D projection of the template to a 3D format. The correspondence depends on the back projection mechanism and method. After the initial alignment of the template model, a 2D-3D correspondence is determined between the defined points of the 2D input contour and the silhouette vertices of the aligned 3D template model for both ML and AP planes, potentially simultaneously.
  • the silhouette vertices may be updated to new positions (target positions) such that their projection, i.e., template projection contour, matches with the input contour.
  • a 2D-2D correspondence between the points of template projection contour points and the input contour points is found.
  • a non-rigid registration approach of SOM may be used instead of rigid registration-based method like ICP technique because the ICP technique can give wrong correspondence for complex contour shapes.
  • the template projection contour points (pp) may be adapted onto the input contour points (pc) using the SOM technique. After the adaptation, the template projection contour points represent the shape of the input contour. The number of the template projection contour points and their topology (connectivity) is preserved in the SOM technique. Hence, the positions of the template projection contour points before and after the adaptation gives the required 2D-2D correspondence.
  • a best matching template projection contour point pp winner—a point nearest to the input contour point— may be determined and its position updated toward the input contour point.
  • the motion of the best matching template projection contour point ppwinner affects a neighbor template projection contour points as well.
  • n(ppwinner, ppm) is an exponential function whose value is high for the template projection contour points that are closer to the ppwinner and small for points which are farther away.
  • the neighborhood function is responsible for topology preservation during the adaptation.
  • the adaptation of all the projection contour points is performed with respect to every input contour point. The adaptation of every template projection contour point and its effect on the neighbor points decrease exponentially.
  • the learning rate l(t) is a function that makes the adaptation process die smoothly with time.
  • the learning rate constant decreases from 0.5 to 0.1.
  • the whole process, including adaptation of template projection contour points with respect to all the input contour points may also be repeated through number of cycles (iterations) until the MAD value between the points of template projection contour and their corresponding closest points of the input contour goes below a threshold, such as 0.15 mm for example.
  • the output of SOM technique is the adapted template projection contour points (ppl) onto the input contour.
  • the template projection contour points before and after the adaptation represents the required 2D-2D correspondence.
  • the 2D-2D correspondence showing which template projection contour point corresponds to which input contour point directly gives the required 2D-3D correspondence of which silhouette vertex of template corresponds to which input contour point.
  • the silhouette vertices may be updated to their target positions in such a way that their projections represent the shape of the input contours.
  • the corresponding target positions vsl of the m th silhouette vertices of the template with initial positions vs are determined using the same 3D-3D point pair calculating method (back projection) used for template alignment as shown in FIG. 5.
  • a projection ray rm is determined starting from the X-ray point source meeting the point pmpl itself.
  • the silhouette vertices may be updated to their target positions, according to which all other vertices of the template are also updated while preserving the overall shape features. This procedure of template deformation is carried out using Laplacian surface
  • a projection and positioning may back- project each of the best matching point(s) to find a position on the back-projected X-ray that is closer to the corresponding silhouette vertices where the target position of each silhouette vertex: 3D-3D correspondence.
  • FIG. 11 illustrates a flowchart of 3D image reconstruction from a single X-ray image. As shown in FIG. 11 A first X-ray is taken keeping the bone in its first pre- determined position with the X-ray source to image distance being known.
  • the first pre-determined position for the first X-ray is such that an anterior-posterior X-ray is taken.
  • a second X-ray is taken keeping the bone in its second pre-determined position with the X-ray source to image distance being known.
  • the second pre-determined position for the second X-ray is such that a medial-lateral X-ray is taken.
  • the second X-ray is orthogonally angularly displaced with respect to the first X-ray, about the axis of the bone.
  • FIG. 12A illustrates an example method of 3D image reconstruction and template deformation separately with respect to ML and then AP X-ray image.
  • FIG. 12B illustrates an example method of the 3D image reconstruction and template deformation simultaneously with respect to ML and then AP X-ray image.
  • FIG. 13 illustrates an example method of determining alignment of the template with respect to the input X-ray image.
  • FIG. 14 illustrates an example method of 3D image reconstruction from a two Orthogonal X-ray image.
  • This invention s 3D deformity correction system and method for elongate bones requires information in terms of anatomical regions, anatomical axes, anatomical landmarks, and anatomical parameters of a 3D deformed bone. Specifically, it may require anatomical landmarks of deformed bone.
  • This system and method is a simulator 19.
  • the 3D model of deformed bone is in the form of mesh (objects with vertices and faces) with triangular elements.
  • the vertices of this mesh represent the points on the surface of the bone (femur or tibia or the like).
  • This deformed bone model can be generated from segmentation of CT scan or reconstructed from multiple X-ray images of deformity.
  • the anatomical landmarks are vertices of deformed bone which represent unique bony features. This can be identified manually with 3D user interface or
  • the landmarks required for calculating anatomical parameters are as follows:
  • the deformity of any elongate bone can be broadly classified into two types: A) Torsional deformity which occurs due to relative twisting between proximal and distal region of the bone; and
  • the bending deformity can be further classified into three types based on the region where deformity occurs:
  • each of the deformities can be calculated by two deformity reference axis defined in a 3D plane called as deformity plane. For any deformity, the extent is measured as the angle between the corresponding two reference axes.
  • deformity plane 3D plane
  • the following table describes the type of deformity, its deformity plane, and the two deformity reference axes:
  • Oblique plane - A plane having normal direction perpendicular to both proximal shaft axis and distal shaft axis
  • the schematic in FIG. 23 shows the method steps of deformity correction. First mid-shaft deformity is corrected followed by proximal and distal joint deformity correction. Finally, the torsional deformity is corrected.
  • a deformed bone can have any combination of the three kind of bending deformities explained above.
  • the goal of bending deformity correction is not only to bring the anatomical parameters in their ideal range but also to bring the mechanical parameters within their ideal range. This goal can be achieved only if all types of bending deformities are corrected, resulting in three bone cuts.
  • the correction of shaft deformity itself can bring the mechanical parameters within their ideal range.
  • the mechanical correction will also bring the cut at same location as shaft cut in this case. This will result in single bone cut.
  • a clinician may prioritise and hence choose to ignore one or more of the bending deformity types according to the extent of deformity within a threshold value.
  • the shaft deformity can be completely ignored and still the mechanical parameters can be brought within its ideal range as shown in schematic FIG. 24. As described in FIG. 24, the following steps are practiced:
  • reference axis 1 will be calculated in the AP plane at an angle LPFA (ideal) with the joint line measured from lateral side passing through femoral ball and reference axis 2 will be calculated in the AP plane at an angle mLDFA (ideal) with the joint line measured from lateral side passing through distal joint centre.
  • reference axis 1 will be calculated in the AP plane at an angle MPT A (ideal) with the joint line measured from medial side passing through proximal joint centre and reference axis 2 will be calculated in the AP plane at an angle LDTA (ideal) with the joint line measured from lateral side passing through distal joint centre.
  • MPT A ideal
  • LDTA ideal
  • One of the joint deformity will be corrected using the deformity reference axes as described earlier (first cut) and the other will be calculated using the new reference axes calculated above (second cut). Hence, this will result in two bone cuts. In most of the cases, this will also lead to correction of shaft deformity.
  • the patient specific instrument is a 3D printed surgical guiding instrument which guides a surgeon to perform operations on bone like resection or drilling.
  • the instrument is designed using 3D bone model in such a way that it sits/fits uniquely onto the patient’s bone surface and has guiding slots for cutting or drilling. Based on the 3D bone model, orientation and position of the cut or drill hole is decided, based on which, the slots for drilling or cutting are designed onto the PSI.
  • This invention describes a method for design of the PSI using 2D X-ray based 3D bone model, for a region (proximal, distal, or mid-shaft) of a bone.
  • the patient specific instrument (PSI) design method requires five inputs:
  • 3D bone model is in the form of mesh (with vertices and faces) with triangular elements.
  • the vertices of this mesh represent the points on the surface of the bone (femur or tibia).
  • This bone model is reconstructed from multiple X-ray images of deformed bone.
  • Region of the bone are subsets of vertices and faces of the 3D bone model which belong to the portion where the PSI is to be placed.
  • Calibration parameters for an X-ray image include position of X-ray source: S(x, y, z), Source Film distance: SFD, principal point position: PPos (X, Y), vectors dirX and dirY representing orientation of the image plane in 3D space.
  • the calibration parameters for AP and ML images are in the same 3D space since a single calibration marker object is used while taking both images. This 3D imaging space is transformed such that the vectors dirX and -dirY for AP image align with XZ plane.
  • the X-ray imaging can be simulated for 3D reconstruction.
  • FIG. 15 shows the imaging space with calibration
  • PSI template for the bone is a 3D mesh (with vertices and faces) with triangular elements.
  • the PSI template design will be specific to a region of a bone.
  • the template is designed manually for a 3D model of an average bone.
  • FIG. 29 illustrates a patient specific instrument.
  • FIG. 30 illustrates the patient specific instrument with a patient bone’s 3D model.
  • FIG. 33 illustrates an external-fixator holds the bone after deformity correction.
  • FIG. 34 illustrates a patient specific instrument, which is automatically designed, to make bone cuts and holes for fixator pins based on simulated external-fixator position of FIG. 30.
  • the anatomical landmarks are vertices of deformed bone which represent unique bony features. These landmarks are calculated based on the standard directions of bone’s Anatomical Coordinate system (ACS), as either extreme points of the 3D bone model in the directions of ACS or fitting standard geometrical shapes like sphere or fitting a 3D plane (a normal vector with a position) on segmented regions of the 3D bone model. The landmarks are calculated for the region for which the PSI is to be placed.
  • ACS Anatomical Coordinate system
  • Anatomical Coordinate System The anatomical parameters defining the deformity in a bone are calculated along standard directions like Anterior-Posterior (AP) and Medial-Lateral (ML) and Superior-Inferior (SI).
  • AP Anterior-Posterior
  • ML Medial-Lateral
  • SI Superior-Inferior
  • a torsional deformity is measured as angle between femoral ball neck axis and posterior condylar axis, along the SI direction.
  • a condylar deformity is measured as an angle between mechanical axis and distal joint line along both AP and ML direction separately.
  • the Z-axis (SI direction) of the Anatomical Coordinate System is along the mechanical axis of the bone.
  • the Y-axis (AP direction) of the Anatomical Coordinate System is along the cross-product of the mechanical axis and the posterior-condylar axis.
  • the X-axis (ML direction) of the ACS is the cross-product of the Y-axis and the Z-axis of the Anatomical Coordinate System.
  • the landmarks can be re-calculated based on the new Anatomical Coordinate System and vice-versa, iteratively. ) Alignment of the Patient Specific Instrument template with respect to the bone region where the Patient Specific Instrument is to be placed.
  • the Patient Specific Instrument template has certain landmarks corresponding to the anatomical landmarks of the bone region where Patient Specific Instrument will be placed.
  • the Patient Specific Instrument template is aligned (rotated and translated) in the coordinate system of the 3D bone model in such a way that the relative orientation (using ICP- Known method to align two 3D point sets) of the anatomical landmarks of the bone region matches the relative orientation of the corresponding landmarks of the Patient Specific Instrument template.
  • the PSI template is then translated along the average normal direction of the bone region in such a way that there will be no intersection between the Patient Specific Instrument template model and the bone model.
  • Average normal direction of a region is the average of all the coordinates of the normal vectors of the faces of a region.
  • the aligned Patient Specific Instrument template is deformed in such a way that the pegs only are translated into certain positions with respect to the bone region while preserving the topology as much as possible.
  • the deformation of the PSI mesh is performed using Laplacian surface deformation (LSD).
  • Laplacian surface deformation smoothly deforms a mesh while bringing a few selected anchor vertices of the mesh to respective target positions (positional constraints) and maintaining the inter- vertices positional relationship (Laplacian constraints) described by Laplacian coordinates (equation xx).
  • the vertices of the pegs portion of the Patient Specific Instrument template are selected as the anchor vertices.
  • the target positions for the anchor vertices are calculated based on the silhouette vertices of the bone region for AP and ML view, determined using respective X-ray image and its calibration parameters.
  • a silhouette vertex of the bone region is selected for each peg portion of the PSI template based on its closest proximity to the centroid position of the vertices of the peg portion.
  • Translation parameters (tx, ty, tz) are calculated as the difference in the coordinates of the selected silhouette vertex and the centroid of the peg region.
  • the target positions mentioned above are calculated as those positions of the vertices of the peg region if they would be translated by the translation parameters.
  • the guiding-slot is a 3D mesh (with vertices and faces) with cylindrical (guiding for drilling hole) or rectangular (guiding for cutting plane) slot within. Based on the orientation and position of the cutting plane or drill holes w.r.t. the bone region, the guiding-slot is placed on the Patient Specific Instrument template. The CSG operation (addition) is then performed between guiding-slot and the Patient Specific Instrument template
  • a patient specific instrument comprises: pegs, guiding slots, and a frame. This is shown, via exemplary embodiments in FIG. 36 and FIG. 37 of the accompanying drawings.
  • pegs are defined as the part of the patient specific instrument mesh that will come in contact with bone surface mesh.
  • Pegs are of three types (FIG. 35):
  • the position of the landing points are in such a way that the landing points will restrict 2 translational degrees and all 3 rotational degrees of freedom of PSI about 3 axis of Anatomical Coordinate System (ACS) (FIG. 30), when PSI is held on bone by surgeon and restricting the remaining 1 translational degree of freedom.
  • ACS Anatomical Coordinate System
  • guiding slots act as a guide for either drilling holes or cutting.
  • a frame connects all the pegs and guiding slot together to form the complete patient specific instrument.
  • the system and method, of this invention provides automatic generation of a patient specific instrument so as to provide pegs’ parameters (the parameters comprising number of pegs, position of pegs, orientation of pegs, shape of pegs, and the like), to provide guiding slots’ parameters, and to provide a frame which connects the pegs (defined by its parameters) and guiding slots (defined by its parameters).
  • pegs parameters comprising number of pegs, position of pegs, orientation of pegs, shape of pegs, and the like
  • guiding slots parameters
  • STEP 1 relates to a step of selection of positions of landing points on the bone mesh.
  • Silhouette positions on the bone mesh with respect to an X-ray source are calculated such that each silhouette position lies on an edge of the bone mesh in such a way that a line from the position of the X-ray source to the silhouette position do not pass through any face of the bone mesh .
  • different set of silhouette positions on the bone mesh can be calculated as follows.
  • SAP-X defined as a subset of the AP silhouette positions lying on part of the mesh faces which is visible from positive and negative direction of X-axis of ACS and lying on the part of the bone mesh where the surgery is performed (usually condyles of bones);
  • SAP-Z defined as a subset of the AP silhouette positions lying on part of the mesh faces which is visible from positive and negative direction of Z-axis of ACS and lying on the part of the bone mesh where the surgery is performed (usually condyles of bones);
  • SML-Y defined as a subset of the ML silhouette positions lying on part of the mesh faces which is visible from positive and negative direction of Y-axis of ACS and lying on the part of the bone mesh where the surgery is performed (usually condyles of bones);
  • SML-Z defined as a subset of the ML silhouette positions lying on part of the mesh faces which is visible from positive and negative direction of Z-axis of ACS and lying on the part of the bone mesh where the surgery is performed (usually condyles of bones).
  • the term,‘visible’, is defined to include mesh faces visible along a particular ACS axis direction are the faces whose outward face normal makes an angle of less than 45 degree with this particular ACS axis direction (FIG. 38).
  • the ACS is a local cartesian coordinate system for a bone and can be determined using the anatomical landmarks/axes.
  • the Z-axis (SI direction) of the ACS is along the mechanical axis of the bone.
  • the Y-axis (AP direction) of the ACS is along the cross-product of the mechanical axis and the posterior-condylar axis.
  • the X-axis (ML direction) of the ACS is the cross-product of the Y-axis and the Z-axis of the ACS.
  • Criteria 1 - The position of landing points are chosen from a subset of silhouette positions on the bone surface in such a way that it satisfies the criterion that the position of the landing points are in such a way that the landing points will restrict 2 translational degrees and all 3 rotational degrees of freedom of PSI about 3 axis of Anatomical Coordinate System (ACS) (FIG. 30), when PSI is held on bone by surgeon and restricting the remaining 1 translational degree of freedom.
  • ACS Anatomical Coordinate System
  • Criteria 2 The position of landing points within a subset is chosen in such a way that at least 2 selected point positions in that particular subset are spread apart at least by an appropriate distance (d a ) calculated for that subset (defined below)
  • first 3 silhouette positions from subset of AP silhouette positions set i.e. (SAP-X or SAP-Z) are selected.
  • d a for subset SAP-X is defined as half of maximum distance measured along Z direction between 2 positions lying in this subset
  • d a for subset SAP-Z is defined as half of maximum distance measured along X direction between 2 positions lying in this subset
  • STEP 2 relates to a step of selection of the peg type (selected from at least a spherical peg type, at least a cylindrical peg type, and / or a cuboidal peg type); which is described in detail, below:
  • the landing position is included in one of the AP silhouette positions set or ML silhouette positions set.
  • the landing position lies in the near- spherical region of the bone mesh like distal condyle of femur.
  • STEP 3 relates to a step of determining orientation and position of the pegs with respect to to its corresponding landing position.
  • Shapes of a peg may be selected from at least a spherical peg position, a cylindrical peg position, and / or a cuboidal peg positon.
  • STEP 4 relates to a step of positioning and orientation of the Guiding-slot.
  • the guiding-slot is a 3D mesh (with vertices and faces) with cylindrical (guiding for drilling hole) or cuboidal (guiding for cutting plane) slot(s) within.
  • the orientation of the axis of cylinder or cuboid and position of the slot(s) is same as the orientation and position of the surgeon or user defined cutting plane or drill holes with respect to the bone mesh.
  • STEP 5 relates to a step of generation of frame mesh.
  • the entire mesh of pegs along with mesh of guiding slot is connected together by a frame mesh.
  • STEP 6 relates to a step of CSG (constructive solid geometry) operation.
  • This step results in CSG Boolean operation of union of the meshes of each of peg, guiding-slot, and frame.
  • This operation also results in combined PSI mesh consisting of new sets of vertices and faces.
  • FIG. 36 illustrates a femur patient specific instrument (PSI).
  • PSI femur patient specific instrument
  • This femur PSI has 5 pegs and 1 additional peg out of which 2 cuboidal pegs correspond to contact points selected from SAP-Z, closest to Distal most landmarks in Lateral and Medial side, 1 cylindrical peg corresponds to contact point from SAP-Z, closest to notch landmark, 1 spherical peg corresponds to contact point from SAP-X at around distal shaft region, and 2 spherical pegs corresponds to contact points SML-Y, at around distal shaft region at sufficient distance apart.
  • 2 cuboidal pegs correspond to contact points selected from SAP-Z, closest to Distal most landmarks in Lateral and Medial side
  • 1 cylindrical peg corresponds to contact point from SAP-Z, closest to notch landmark
  • 1 spherical peg corresponds to contact point from SAP-X at around distal shaft region
  • 2 spherical pegs corresponds to contact points S
  • FIG. 37 illustrates a tibia patient specific instrument (PSI).
  • PSI tibia patient specific instrument
  • This tibia PSI has 5 pegs; 3 spherical pegs corresponding to contact points selected from SAP-Z, at the plateau region (undercut/concave surface) and 2 spherical pegs corresponding to contact points selected from SML- Y, at proximal anterior region.
  • Example systems and methods may include view manipulation to manipulate views of the rendered and deformed 3D template. This may enable a user to carry out any or more of: rotate, pan, zoom the view using touch based user input;
  • a user or a surgeon may virtually plan a surgery using the manipulate views of the rendered and deformed 3-dimensional template.
  • Surgery planning tools may allow the user or the surgeon to plan the surgery, virtually.
  • a surgeon can now use the 3D view of the bone/joint anatomy to plan certain surgeries by manipulating the 3D bone models or importing 3D models of bone implants (depending on the surgery) onto the rendered image.
  • the manipulations May include: rotate/translate the 3D bone model about/along all the 3 axes of the Cartesian coordinate system using touch inputs; resect/Cut the bone into segments and rotate or translate the individual segments using various options provided; select the landmark points (regions) on the 3D bone surface; and/or import 3D models (in STL format) of bone implants onto the 3D interface of the software application.
  • Example systems and methods may thus enhance portability.
  • Conventional process of planning the surgery use hard copies of X-ray image of the particular region of the patient's body which has to be operated and does not allow a surgeon to simulate the post-operative conditions and it is inconvenient for measurements.
  • Example embodiments and methods use digital X-ray images that can be handled on a portable tablet; a portable method of surgery planning where the surgery plan/simulation can be easily referred during the surgery in the operation theatre.
  • Example systems and methods allow planning of the surgery in 3D view of bone/joint anatomy, which requires only 2-dimensional X-ray images of a patient.
  • Prior art techniques to obtain a 3D model of bones uses CT scans as input and patient has to undergo CT scanning.
  • example systems and methods require only low cost 2D X-ray images which have about 20 times less cost than a CT scan
  • the input X-ray images can be acquired by the normal routine procedure of X-ray images with conventional single view imaging equipment; biplanar X-ray imaging equipment or exact orthogonal views of images are not required; 2D X- ray images have around 500 times less radiation than CT scans, lessening patient exposure; 2D X-ray imaging equipment is more prevalent and less expensive than CT scan equipment; and CT scan data is much larger, complicating handling and communication.
  • example systems and methods on smaller or tablet devices helps in accurate planning/simulation of the surgery; the tablet interface enables a portable process with a touch-based user interface with easier interactive, touch-based 3D view manipulation of 3D models and views. Case studies can be easily saved in the mobile tablet device and can be shared and archived and 3D models can be printed. Example methods of 2D to 3D conversion based on Laplacian deformation may provide a more efficient shape deformation technique.
  • Example methods may be used in combination and/or repetitively to produce multiple options and functionalities for users of communications devices.
  • Example methods may be performed through proper computer programming or hardware configuring of networks and communications devices to receive augmented reality, origin, and limitation information and act in accordance with example methods, at any number of different processor-based devices that are communicatively connected.
  • example methods may be embodied on non-transitory computer-readable media that directly instruct computer processors to execute example methods and/or, through installation in memory operable in conjunction with a processor and user interface, configure general-purpose computers having the same into specific communications machines that execute example methods.

Landscapes

  • Health & Medical Sciences (AREA)
  • Surgery (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Veterinary Medicine (AREA)
  • Public Health (AREA)
  • Orthopedic Medicine & Surgery (AREA)
  • Dentistry (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Robotics (AREA)
  • Physical Education & Sports Medicine (AREA)
  • Transplantation (AREA)
  • Surgical Instruments (AREA)

Abstract

L'invention concerne un procédé pour l'obtention de modèles d'instruments pour interventions osseuses propres à un patient, ledit procédé consistant à : sélectionner les positions de points d'arrivée sur le maillage osseux ; calculer les positions de contours sur le maillage osseux ; obtenir différents ensembles de positions de contours sur le maillage osseux pour différentes positions d'une source d'imagerie, l'ensemble des positions de contours étant sélectionné parmi un ensemble de positions de contours AP et/ou un ensemble de positions de contours ML, lesdits ensembles de contours étant divisés en quatre sous-ensembles de contours sélectionnés à partir d'un groupe d'un premier sous-ensemble de contours, d'un deuxième sous-ensemble de contours, d'un troisième sous-ensemble de contours et d'un quatrième sous-ensemble de contours ; sélectionner un type de clou sur la base de la position d'arrivée ; déterminer l'orientation et la position des clous sélectionnés par rapport à la position d'arrivée qui leur correspond ; déterminer l'orientation et la position d'au moins une fente de guidage ; et générer un maillage cadre de telle sorte que le maillage des clous et le maillage de la fente de guidage soient reliés par ledit maillage cadre ainsi généré.
PCT/IN2019/050233 2018-03-21 2019-03-21 Systèmes et procédés pour l'obtention de modèles d'instruments propres à un patient WO2019180747A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
IN201821010313 2018-03-21
IN201821010313 2018-03-21

Publications (1)

Publication Number Publication Date
WO2019180747A1 true WO2019180747A1 (fr) 2019-09-26

Family

ID=67986871

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IN2019/050233 WO2019180747A1 (fr) 2018-03-21 2019-03-21 Systèmes et procédés pour l'obtention de modèles d'instruments propres à un patient

Country Status (1)

Country Link
WO (1) WO2019180747A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111956318A (zh) * 2020-07-07 2020-11-20 济南大学 定位导板及其制作方法、定位导板模型生成方法及***
CN113679447A (zh) * 2021-07-20 2021-11-23 国家康复辅具研究中心 一种用于股骨远端截骨术的导航模板及其设计方法
CN116712171A (zh) * 2023-08-11 2023-09-08 北京维卓致远医疗科技发展有限责任公司 一种粗隆间骨折导航方法、设备及可存储介质

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160270855A1 (en) * 2007-05-14 2016-09-22 Queen's University At Kingston Patient-Specific Surgical Guidance Tool and Method of Use
US20170112628A1 (en) * 2015-10-22 2017-04-27 DePuy Synthes Products, Inc. Customized patient-specific orthopaedic instruments for component placement in a total hip arthroplasty

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160270855A1 (en) * 2007-05-14 2016-09-22 Queen's University At Kingston Patient-Specific Surgical Guidance Tool and Method of Use
US20170112628A1 (en) * 2015-10-22 2017-04-27 DePuy Synthes Products, Inc. Customized patient-specific orthopaedic instruments for component placement in a total hip arthroplasty

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111956318A (zh) * 2020-07-07 2020-11-20 济南大学 定位导板及其制作方法、定位导板模型生成方法及***
CN113679447A (zh) * 2021-07-20 2021-11-23 国家康复辅具研究中心 一种用于股骨远端截骨术的导航模板及其设计方法
CN113679447B (zh) * 2021-07-20 2023-02-28 国家康复辅具研究中心 一种用于股骨远端截骨术的导航模板及其设计方法
CN116712171A (zh) * 2023-08-11 2023-09-08 北京维卓致远医疗科技发展有限责任公司 一种粗隆间骨折导航方法、设备及可存储介质
CN116712171B (zh) * 2023-08-11 2023-11-03 北京维卓致远医疗科技发展有限责任公司 一种粗隆间骨折导航方法、设备及可存储介质

Similar Documents

Publication Publication Date Title
US10217217B2 (en) Systems and methods for obtaining 3-D images from X-ray information
US20210012492A1 (en) Systems and methods for obtaining 3-d images from x-ray information for deformed elongate bones
US20210007806A1 (en) A method for obtaining 3-d deformity correction for bones
US11826111B2 (en) Surgical navigation of the hip using fluoroscopy and tracking sensors
US7394946B2 (en) Method for automatically mapping of geometric objects in digital medical images
EP1598778B1 (fr) Méthode de positionnement automatique d'objets géometriques dans des images médicales
US9345551B2 (en) Implant design analysis suite
JP2020175184A (ja) ２ｄ解剖学的画像から３ｄ解剖学的画像を再構成するシステムおよび方法
CN110189352B (zh) 一种基于口腔cbct图像的牙根提取方法
Karade et al. 3D femur model reconstruction from biplane X-ray images: a novel method based on Laplacian surface deformation
WO2019180746A1 (fr) Procédé d'obtention de correction de malformation tridimensionnelle pour os
US11257241B2 (en) System and method for component positioning by registering a 3D patient model to an intra-operative image
WO2023078309A1 (fr) Procédé et appareil d'extraction de point caractéristique cible, dispositif informatique et support de stockage
WO2019180747A1 (fr) Systèmes et procédés pour l'obtention de modèles d'instruments propres à un patient
US20230071033A1 (en) Method for obtaining a ct-like representation and virtual x-ray images in arbitrary views from a two-dimensional x-ray image
US20220156942A1 (en) Closed surface fitting for segmentation of orthopedic medical image data
Ün et al. An analytical method to create patient-specific deformed bone models using X-ray images and a healthy bone model
US20240185509A1 (en) 3d reconstruction of anatomical images
US20230263498A1 (en) System and methods for calibration of x-ray images
Zsemlye Shape prediction from partial information
El Dakhakhni Reconstruction of Patient-Specific Bone Models from X-Ray Radiography
Guéziec et al. Exploiting 2-D to 3-D intra-operative image registration for qualitative evaluations and post-operative simulations

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19771602

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19771602

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 19771602

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 04/06/2021)

122 Ep: pct application non-entry in european phase

Ref document number: 19771602

Country of ref document: EP

Kind code of ref document: A1