WO2023181072A1 - Digital system and 3d tool for training and medical counselling in ophthalmology - Google Patents

Abstract

The invention is about a creative digital system and 3D tool developed on an advanced interactive 3D touch interface, using patient's real-time confocal fundus images along with their multimodal images in multiple sections and which is capable of being viewed in 3D format. (Fig.5) The invention is capable of producing models with clear visibility up to the level of electron microscopic view, the standard models are viewed for the study of Ophthalmology for training of students and also for medical counselling of patients about the anatomy (Fig 2(236)) and pathology (Fig 2(237)) condition of eye, the true eye fundus image (Fig 2 (221)) of the patient is super imposed on 3D models to get near accurate 3D models true to the eye conditions, the digital system can be integrated with any of the state of art technologies such as standard 3D tool (Fig 2 (229)) from a PC, mobile or any other compatible device, alternately the 3D image can also be viewed through AR, VR compatible devices also, the digital system is also capable of 3D printing (Fig 2 (235)) for objectified study, training and counselling of Ophthalmology.

Description

FORM 2

THE PATENT ACT 1970

(39 of 1970)

&

The Patent Rules 2003

COMPLETE SPECIFICATION

{Section 10 and Rule 13}

TITLE OF THE INVENTION

“DIGITAL SYSTEM AND 3D TOOL FOR TRAINING AND MEDICAL COUNSELLING IN OPHTHALMOLOGY”

Applicant

DR Prasanna Venkatesh Ramesh

MAHATHMA CENTRE OF MOVING IMAGES PRIVATE LIMITED., No: # 06, SESHAPURAM, TENNUR, TIRUCHIRAPPALLI - 620017, TAMIL NADU, INDIA

(INDIAN BODY SMALL COMPANY)

The following specification particularly describes the invention and the manner in which it is to be performed. DIGITAL SYSTEM AND 3D TOOL FOR TRAINING AND MEDICAL

COUNSELLING IN OPHTHALMOLOGY

FIELD OF THE INVENTION

The invention is about developing a digital system and tool for studying Anatomy & Pathology of human organs. In particular the present embodiment of the invention facilitates training to the students of ophthalmology and also enables medical counselling for the eye patients in a much more factual, fun and easy to access way which otherwise is not possible.

BACKGROUND OF THE INVENTION

The eye is a complex structure, where human eye dissection for students is not practical. The concepts pertaining to ophthalmology have lots of theoretical frameworks and the young students and novice surgeons may have to mentally visualize these concepts, during the initial days of training. This invention presents a software I application as well as 3D ophthalmic structures, which facilitates learning ophthalmology for students in a much more factual, fun, and easy to access way.

The dissection of the human eye, is an impractical method to teach the human eye anatomy to students. We can only use an animal eye such as a horse or goat which also is not very effective because as soon as the animal dies, the eyeball starts to shrink and hence not very effective in teaching. The eye itself is a small organ, which has extremely tiny microscopic structures, some of which cannot even be made visible to teach the students. For example (The Trabecular Meshwork).

Throughout the last decade, several innovative and interactive digital and smartphone applications became useful learning resources for ophthalmologists. They teach anatomy, clinical ophthalmology, mnemonics, grading systems, and surgical skills. However, in most of the anatomy teaching apps, the images used are 2D animated images or 3D animated models, which are quite different from the real-time images, that one encounters in clinical practice. Also, there are very few zoom sections of complex, important landmark structures such as the angle of the anterior chamber in those teaching applications.

With the fast technological advancements, interactive digital concepts such as Augmented reality (AR), Virtual reality (VR) and 3D printing is slowly and steadily becoming a substantial part of modern life and more so with the increasing applications in the field of medicine and ophthalmology. Innovative ways to teach and learn e-ophthalmology are in high demand. This pedagogical transformation in e-ophthalmology aims to reinvent the approach to ophthalmic teaching through virtual platforms.

Hence there is a need for smart teaching methods to help students of ophthalmology for proper understanding of the anatomy and pathology of eye in the absence of dissection or any other objectified training aids to make the study more interested and fun filled like a gaming application.

Present invention is aimed at development of a digital system and 3D tool for training and medical counselling in ophthalmology and as the capability to integrate with virtual simulators for training of specialized physicians, mainly in surgical education

PRIOR ARTS . Indian Patent Application - 201841008545

A computer implemented system for analyzing a fundus image of a patient is disclosed. The system comprises at least one processor coupled to a non-transitory computer readable storage medium configured to store a fundus image analysis application , comprising: a graphical user interface comprising interactive elements configured to enable capture and analysis of the fundus image; a reception means adapted to receive an input from an image capturing device based on a plurality of parameters of the image capturing device; an interactive fundus image rendering means adapted to dynamically render the input; a fundus image capture means adapted to capture the fundus image based on the dynamically rendered input; a first 10 analysis means configured to generate a first label for the fundus image; a second analysis means configured to generate a second label for the fundus image.

2. Indian Patent Application - 201934019822

An apparatus for producing a fundus image includes: a processor and a memory; an illumination component including a light source; a camera; and a display, the memory stores instructions that, when executed by the processor, cause the apparatus^ to: display a target element on the display; display a light source reflection from a cornea of an eye on the display; update a position of the light source reflection on the display as a caregiver manipulates a position of the apparatus relative to the eye; as time elapses, modify the display to allow the light source reflection to more easily be positioned within the target element; and automatically initiate fundus image captures the camera when the light source reflection is within the target element on the display.

3. Indian Patent application - 202047005186

An adaptor for attachment to an image acquisition device, the image acquisition device having one or more camera apertures. The adaptor has a housing defining a passage along which light waves may travel, an objective lens arrangement within the passage, a secondary lens arrangement within the passage positioned such that, when the adaptor is attached to an image acquisition device, the secondary lens arrangement is along a possible light pathway between the objective lens arrangement and one or more camera apertures. The lens arrangements are together configured to magnify an image of a pupil of the eye in proximity to a plane of one or more camera apertures and to focus light waves from a light source at a point external of the adaptor and offset from the optical axis of the objective lens arrangement .

4. Indian Patent Application - 202141026746

This invention emphasis on detection of Glaucoma based on Cup to Disc Ratio and ISNT rule. The proposed project involves pre-processing, optic cup segmentation, optic disc segmentation and evaluation of CDR and ISNT. The OD is segmented from the fundus images obtained from RIM- ONE database using segmentation techniques such as mean, global and Otsu thresholding. The OC is segmented using Otsu thresholding and K- means clustering. The results of segmentation are compared with ground truth using Jaccard similarity measures and the best segmented result is obtained. The segmented OD and OC are analysed for CDR and ISNT rule. Based on the results of the CDR and ISNT rule evaluation the retinal images are classified as normal or abnormal and the stages of severity of glaucoma are identified.

5. Indian Patent Application 202141043592 Diabetic retinopathy is a complication of diabetes that affects the eyes. It is caused by blood vessel damage to the photosensitive tissue at the back of the eye (retina). At first, diabetic retinopathy may cause no symptoms or only minor vision problems. Eventually, it may cause blindness. Compared with the number of patients in India, the number of doctors is quite small, causing delays in the diagnosis of various diseases. However, the late diagnosis of diabetic retinopathy can cause irreversible damage to the eyes, resulting in its total and permanent blindness. This disease is treatable, but its damage is not completely reversible. To avoid this situation, we decided to use machine learning to automate the diagnosis process. The increase in 4,444 cases of diabetes limits the ability of the current 4,444 manual tests. New algorithms for assisting diagnosis are becoming very important today. Early detection of diabetes can help all patients and limit negative health consequences such as blindness, so we use the support vector machine (SVM) algorithm to classify the extracted histogram. A histogram grouping scheme is proposed to represent features. Experimental results show that LESH is the best performing technology, and the accuracy obtained by using SVM with radial basis function kernel (SVMRBF) is 0.904.

6. US Patent - US7593559

A method/system preserves annotations of different pathological conditions or changes that are recognized on cross sections within a three-dimensional Volume of a patient’s eye so that the annotations are maintained in a visible state in an enhanced projection produced with a SVP technique, it is thus possible to co register the annotated conditions or changes with other types of two dimensional face images such as images from other ophthalmic devices (e.g., angiography device, microperimetry device, autofluorescence device, fun dal photography device.). The annotations are also maintained in a visible state in the co registered image.

7. US Patent - US20090010507A1

A system and method are provided for generating a three-dimensional (3D) model of an anatomical structure of a patient using a plurality of two dimensional (2D) images acquired using a camera. The method includes the operation of searching the plurality of 2D images to detect correspondence points of image features across at least two images. Camera motion parameters can be determined using the correspondence points for a sequence of at least two images taken at different locations by the camera moving within the internal anatomical structure. A further operation is computing dense stereo maps for 2D image pairs that are temporally adjacent. A consistent 3D model can be formed by fusing together multiple 2D images which are applied to a plurality of integrated 3D model segments. Then the 3D model of the patient's internal anatomical structure can be displayed to a user on a display device. 8. US Patent - US20210209724A1

To provide an image processing device capable of observing the inside of the eye in a wider range and simultaneously. There is provided an image processing device including an integration processing unit configured to integrate ophthalmologic images acquired by a plurality of types of ophthalmologic image capturing devices by correlating intraocular positions and to generate one integrated image indicating intraocular information in a range wider than a range indicated by each of the ophthalmologic images.

DRAWBACKS PRESENT IN PRIOR ART

In most of the anatomy teaching apps prevailing, the images used are 2D animated images or 3D animated models, which are quite different from the real-time images, that one encounters in clinical practice. Also, there are very few zoom sections of complex and important landmark structures such as the angle of the anterior chamber in those teaching applications.

Even in all the 3D models of the structures of eye done so far, they fail to display structures present in the Angle of Anterior Chamber, which is being very important structure in the eye which when affected, leads to change in intra ocular pressure thereby leading to various conditions including Glaucoma.

There are also limitations present in the previously created 3D models while capturing retina/fundus etc., It is mostly be hand drawn images or computer drawn/edited images, which would almost never give the experience of real fundus image and even if we get the real fundus image, it would require a great amount of effort and labor for teaching and training.

There are no good tools other than images in different angles, to teach the students about certain important and complex vascular systems such as the cerebral venous system.

Hence there is a need for a Digital System and 3D tool which must be very helpful in teaching the students about the various structures involved in Ophthalmology and such tactile form of teaching which is rather rare is aimed to improve their knowledge greatly.

NECESSITY FOR NEW INVENTION

Medical ophthalmic learning concept for studying ocular anatomy and pathophysiology has attracted much attention. There is always need felt for a system and tool to be developed on an advanced interactive 3D touch interface, using patient’s real-time confocal fundus images along with their multimodal images in multiple sections and which is capable of being viewed in 3D format by any of the interfaces such as, computer system, mobile device etc., by utilizing the technologies in creative 3D applications.

The digital system and the 3D models created by the invention is capable of integrated through inventive digital platforms and 3D models for enhanced version of the real physical world that is achieved through the use of digital visual elements, sound, or other sensory stimuli delivered via technology like Augmented Reality (AR), Virtual Reality (VR) and further for objectified study of the ophthalmology through 3D digital printing for objectified physical models.

The invention is capable displaying true 3D image for every normal or pathological fundus, with a simple click of a button, various multimodal realtime images such as autofluorescence and infrared imaging can be visualized three-dimensionally.

The invention not only benefit the residents and ophthalmologists, but also will help in customized patient counselling, as the digital system application is also constructed in a patient-friendly format. Hence, the nature of the disease and procedures ranging from, a simple YAG laser peripheral iridotomy or anti-VEGF injection to a complex trabeculectomy/retinal surgery can be counselled with this app. The complex structures such as the angle of the anterior chamber, ciliary body, and posterior segment of the eye are constructed with 3D photoreal visuals of real-time images for simple and effective patient counselling experiences.

In addition to these, other anatomical structures such as the cerebral nervous systems, Extra Ocular Muscles, Cerebral vascular systems pertaining to the eye, Brain stem nuclei and other systems such as the lacrimal systems could also help in better understanding for the students and would perfectly suit patient counselling.

OBJECTS OF THIS INVENTION

The principal object of the invention is to develop a digital system and 3D tool to facilitate a better way of teaching & patient counselling in the field of Ophthalmology and other fields.

Another object of the invention is to make sure that the structures of the Eye are portrayed correctly, up to the electron microscopic view.

Yet another object of the invention is to recreate the whole eye in 3D, which would appear as if it is alive, thereby making sure the students will be able to see the Anatomy/Physiology of a live eye.

Yet another object of the invention is to make sure that the students are able to see the different pathologies present in the real fundus, exactly as how an experienced Ophthalmologist would see using the slit lamp but in a 3- dimensional view.

Yet another object is to make sure students are able to learn other structures related to the eye and learn them in a gamified way, such as a puzzle solving, thereby adding more fun into learning, as well as giving them the knowledge of very complex anatomical structures. This could be achieved in both software as well as real toys. Yet another object of the invention is to enable patient counselling by educating them with pathology condition of their eye through the 3D models from the digital system by viewing with aid of 3D viewing tools.

Yet another object is to develop the tool which is capable of integrating with AR, VR and capable producing objectification of the ophthalmology studies by 3D printing technique

Yet another object of the invention is to provide accurate 3D models for surgical simulation training.

Another object of the invention is to integrate the digital data of the models with electronic medical record of the patients

Yet another object of the invention is to develop similar Digital System and tool for medical and other fields for training, counselling and simulation purposes.

STATEMENT OF THE INVENTION

The invention is about development of digital system and tool for pedagogical transformation in e-ophthalmology and aims at reinventing the approach to ophthalmic teaching through virtual platforms. This 3D platform and tool is constructed with pioneering ways of cult teaching, which can be extended out to any video output device, along with photoreal cinematic 3D learning in ophthalmology. Multimodal confocal real-time imaging and ultra- zoom features in an e-learning app with deep visualization experiences have never been reported in the literature and can pave the way for a new age in ophthalmic pedagogy. The System is capable of being integrated with multiple digital infrastructures such as a PC, Mobile and similar devices and also can be integrated with any of the viewing tools such as augmented reality, virtual reality or even enables 3D printing for objectified learning methods in the field of Ophthalmology

BRIEF DESCRIPTION OF DRAWINGS

The accompanying drawings describe the invention in greater detail containing same reference numbers across figures to refer to like parts, devices, equipment, or components.

The invention, being digital system and 3D tool for training and medical counselling in Ophthalmology, its major processes from input to output and the sequential work flow is illustrated in the block diagram of Figure 1.

Figure 2. Illustrates the process flow chart of the entire inventive concept comprising of digital system along with various applications, tools, and devices.

Figure 3. Illustrates 3 D modelling process in a flow chart which produces the desired output models. Figure 4. Illustrates the flow chart depicting various viewing tools, digital applications and devices which produces the respective output forms

Figure 5. Illustrates sequential flow chart with image representation (screen shots) for conversion of 2D Fundus Images into 3D Fundus Models.

Figure 6. Illustrates the sequential process of texturing of the Eye Model through image representation (screen shots)

Figure 7. Illustrates screen shots with annotations of Type 1 3D modelling process output for Cerebral Venous System (CVS) - (7A) and Cerebral Arterial System (CAS) - (7B) respectively.

Figure 8. Illustrates screen shots with annotations of Type 1 3D modelling process outputs for Optic pathway

Figure 9. Illustrates screen shots with annotations of Type 1 3D modelling process outputs for Brain Stem Nucleus.

Figure 10. Illustrates screen shots of Type 2 3D Modelling process output with annotations for Extra Ocular Muscles (EOM) of Eye, Brain, and Eye Ball respectively.

Figure 11. Illustrates screen shots of 3D viewing of Anatomy of Eye Ball with detailed annotations of various parts. Figure 12. Illustrates screen shots of 3D viewing of Eye Ball with various pathological conditions of Eye ball.

DESCRIPTION OF THE INVENTION

The invention is about development of a Digital system and 3D tool for training and medical counselling in ophthalmology. The development of digital system comprises of digitally created eye model with necessary detailing for most appealing cross-sectional view of the eye fundus (Fig. 5 (510)) The creative work is developed with the guidance of ophthalmologists, in such a way that the posterior pole of the retina (Fig 10A) , as well as the cross-sectional view of the eye and all parts of the eye (Fig 10.C & Fig 11) , Cerebral Venous system (CVS) (Fig 7A) and Cerebral Arterial System (CAS)(Fig 7B) etc., are clearly visible up to the level of electron microscopic view.

The major process of the inventive concept is illustrated in Figure 1 (100 to 117), it comprises three stages, such as Input (Fig.1. (100)), the process (Fig.1. 101)), and the output (Fig.1. (102)). The input stage comprises sourcing of 2D images (Fig.1 (105)), from fundus scanner (Fig.1 (103)) and from other sources (Fig.1. 104)) for creation of eyeball and other models.

The processing stage (Figi (101) comprises core activity of the invention, which comprises of 3D modelling process (Fig1(107)) for creating 3D models (Fig1(108)), further integration with digital tools and applications (Figi (109)) for viewing of 3D digital images and also integration with analog process methods (Figi (110)) for 3D printing respectively.

The output (Fig1(102)) of the invention is obtained by integrating the 3D models with digital tools (Fig1(108)) and analog process (Fig1(109)) respectively. The plurality of outputs obtained through digital tool comprises of 3D image viewing of Anatomy (Figl(ll l)), Pathology (Fig1(112)), viewing of animation videos (Fig1(113)), integration of 3D models for surgical simulations (Figi (114)) and integration to patient’s medical records (Fig1(115)) respectively. The 3D models (Fig1(107)) integration with Analog process methods (Fig1(109)) produces objectified physical models through 3D printing (Fig1(116)). The integration of the 3D models (Fig 1(107)) with customized software applications (Fig1(110)) aids viewing of anatomy (Figi (111 )) and pathology (Figi (112)) of the 3D models through digital viewing tools.

Figure 2. (200 to 251 ) Illustrates the detailed process of the entire invention comprising of digital systems, integrated with various application tools and devices. 3D modelling is the core process of the invention, for the development of the digital system and tools for training and medical counselling in ophthalmology.

The plurality of 3D modelling process (Fig 2(200)) comprises

Type (1) modelling process (Fig 2(201)) for creation of models of smaller blood vessels and necessary systems and Type (2) modelling process

(Fig 2(202)) is for whole organs.

The process methods for Type 1 (Fig 2 (201)) modelling process comprises of: - i) Drawing the 3D models (Fig 2(203)) in the software tool; ii) Extrusion in a circular form (Fig 2(204)) along the path of the drawing ; iii) Adjustment of thickness (Fig 2(206)) to resemble the natural shape and size of the respective model; iv) Creation of colorless model (Fig 2 (206)) of the desired structure; v) Application of new material to the model (Fig 2 (208)); vi) Assignment of appropriate color on the material by choosing from the default color palette (Fig 2 (209)); and vii) Creation of the Typel 3D model outputs (Fig 2 (210)), comprising of plurality of models of Cerebral Arterial System (CAS) (Fig 2(211)), Cerebral Venous System (CVS) (Fig 2 (212)), Cerebral Nervous System (CNS) (Fig 2 (213)), Brain Stem Nucleus (Fig 2 (214)) and Optic Pathway (Fig 2 (215)) and many other new models of subject matter of interest and provide an avenue for studying Ophthalmology Students and for patient counselling.

The methods for Type (2) modelling process (Fig 2(202)) comprises of creating of models of whole organs, the process comprises of i) Choosing the basic shape of the organ (Fig 2 (218)), from the software application; ii) Molding into desired shape (Fig 2 (217)) like it is done in the case of clay models and create the shape of the desired model. For example, the default sphere shape suits for creating eye ball; iii) Create colorless model (Fig 2 (206)) and further subject it to texturing process (Fig 2 (218)); iv)A new material is created from the software application and is subjected to UV unwrapping process (Fig 2(219)) pertaining to respective models and texture; v) Appropriate image is created that matches unwrapped UV layout and then the image is applied as texture (Fig 2 (220)); and vi)The process of superimposing the textured image on to the unwrapped UV is called UV mapping and this way creates accurate detailing on the model up to the microscopic level of the actual organ, the output of Type (2) modelling process (Fig 2 (202)) comprises plurality of models comprising eyeball (Fig 2 (224)), Brain (Fig 2(226)), Skin (Fig 2 (227)), Skull (Fig 2 (228)) and like. Fig (6) (600 to 607) illustrates the texture application and model creation process for eye ball.

The Type 2 modelling process also allows super imposing of the patient images captured from fundus scanner (Fig 2 (221)) on to the standard 3D models to create near actual conditions of pathology of the patient’s eye and thus aids for counselling and prescription of appropriate treatment Figure (5) (500 to 510). Illustrates the conversion process of 2D confocal images from the fundus scanner into appropriate 3D fundus model. It is applied to create anatomy and pathological conditions of the eyeball.

Once the models are created, they can be viewed through viewing tools (Fig 2 (228)), there are multiple options for viewing from the plurality of available digital viewing tools and applications (Fig 2 (229)) and also there is additional option for producing analog objects (Fig 2(230)) from the standard 3D image from the system by 3D printing (Fig 2 (235)) process. The plurality of digital viewing tools comprises of Eye MG 3D (Fig 2(231)), Eye MG AR (Fig 2 (232)), Eye MG Max (Fig 2(233)) and Eye MG Holo (Fig 2(234)) are developed for specific application and are generally tested for its performance. As the research & development being continuously evolving additional viewing tools may also be inducted within the main scope of the invention being Digital System and 3D tool for Training, Diagnosis, and medical counselling in Ophthalmology.

Eye MG 3D (Fig 2 (231)) has got two models one for PC’s and another for Android devices. The app enables to view the anatomy (Fig 2 (236)) and pathology of eye. Anatomy modules shows the annotated views of the different parts of the eye (Fig 2 (239 to 243)) and also has the options for zooming and focusing on specific part of the particular organ through pre-programmed viewing pattern aided by the various tabs of the app makes the viewing experience of actual organ and its parts along with pathological conditions of different diseases that affect the fundus of eye close to the microscopic view.

The viewing of Anatomy has the option to view with lens or without the lens and with or without presence of vitreous liquid based on the user preference. On the other hand, pathology conditions of the eye can be viewed in color mode, auto-florescence mode, or infra-red mode (Fig 2 (249)) respectively as per specific requirement of the study.

Eye MG AR (Fig 2 (232)) is another version of Eye MG 3D app which utilizes augmented reality technology and is designed for Android devices to view pathology models only (Fig 2 (237)) and requires AR template for viewing. When the app is started, it opens the camera of the android device with augmented reality enablement. AR template is a coded Image where the program triggers when scanned by the app’s camera. As soon as the image is scanned by the android device through the app, it automatically generates the 3D model onto the template thereby giving an experience as if the 3D model is actually present on the AR template. Thus, the user can experience the views of pathology features from Eye MG 3D in augmented reality. Eye MG Max (Fig 2 (233)). The application is developed for iPhones and has all the features of Eye MG 3D (Fig 2(231)) and Eye MG AR (Fig

2(232)) but also has some additional features.

Eye MG Holo (Fig 2((234)), The app is developed for HoloLens 2 device. This device is of a cutting-edge technology has features including spatial recognition, hand recognition, iris tracking etc. The application has multiple features including anatomy and pathology modules. Other features include surgical simulations (Fig 2((251)) and integration with Electronic Medical records (Fig 2((250)) and like.

The Surgical Simulations (Fig 2 (251)) is a device or model used for training individuals by imitating situations they will encounter in real life. Surgical simulators, such as human cadavers, live animals, bench-top models, and virtual reality (VR) systems recreate surgical situations for trainees to practice and hone their skills. The invention through its accurate modeling process is able to create near real organs and ensures the student surgeons are able to undertake stimulatory surgical practices with the digital models, as the dissection of the human eye, is an impractical method to teach the human eye surgery to students. Hence the invention provides additional avenue for surgical simulations for the training purposes by integrating accurate 3D pathological conditions (Fig 2 (248)) and models of the eye being created by the inventive concept with the simulators and to train medical students and surgeons in specific types of procedures without the use of animals or cadavers before working with live patients. The Systems can enable the surgical simulations (Fig 2 (251)) for impairments such as, i) Trabeculectomy; ii) Foreign Body Removal; iii) Stent Placement; iv) Cataract Surgery v) Intra Ocular Foreign Body Removal vi) Retinal Detachment vii) Vitrectomy viii) Peripheral Iridectomy etc.,

The Integration with Electronic Medical records (Fig 2((249)) digitizes the health care process. Patient on arrival to hospital undergo examination with necessary scanning and diagnosis and the screening details gets stored into the data base. Eye MG Holo (Fig 2((234)) fetches the data from this data base and converts the scanned 2D Images into 3D models of the eye for further analysis and study of the patients.

The Analog product is made either through 3D printing or through injection molding. These are the 3D modelling as we see digitally but make a real physical toy like model which can be held by hand. These tools would also be of a real resource for students as they would be viewing the structure like never before.

Figure 3 illustrates the process stage of the invention (Fig 3(300)), the process comprises of Type (1) (Fig 3(301)) and Type (2) (Fig 3(302)) modelling process and their various output models (Fig 3(310)) the outputs are seamlessly integrated with plurality of digital tools (Fig 3(328)) and applications (Fig 3(331)) to view the anatomy (Fig 3(332)) and pathology (Fig 3(333)) of the eye ball and other organs. The digital tools (Fig 3(328)) are further integrated to produce plurality of other output options such as displaying animated videos (Fig 3(334)) of the anatomy and pathology, aid in surgical simulation (Fig 3(335)) and also digitize health care facilities by integrating patient’s medical records (Fig 3(336)) with their scanned models. The analog integration (Fig 3(329)) produces 3D physical objects similar to clay models through 3D printing (Fig 3(337)).

Fig 4 (400 to 434) illustrates 3D viewing and details of the various outputs as viewed from different viewing tools.

Fig 5 (500 to 510) displays screenshots in a sequential process flow illustrating the conversion process of the of 2D confocal image to 3D fundus model. The patient details are scanned by the fundus scanner (Fig 5 (500)) to capture the 2D confocal image (Fig 5(501)) of the eye. Now the UV layout (Fig 5 (503)) related to the fundus image is created in the modelling platform (Fig 5 (502)) (software) and applied over the 2D confocal image (Fig 5(501)) for super imposing of the fundus image with UV layout (Fig 5 (504)). Thus, the UV layout gets mapped with fundus image (Fig 5 (505)) and automatically applied (Fig 5 (509)) onto the pre-created plain 3D fundus models (Fig 5 (508)). Thus, the 3D textured fundus digital model (Fig 5 (510)) is obtained. Similar process is applied for creation of plurality of other 3D models like brain (Fig 3(325)), Skull (Fig 3(326)), Skin (Fig 3(327)) etc.,

Fig 6 (600 to 607) illustrates the process of applying texture to colorless models (Fig 2(208)) of Type (2) 3D modeling process (Fig 2(202)). The plurality of pre-created textured images (Fig 6(600) like textured images of fundus (Fig 6 (601)), sclera (Fig 6(602)), Optic Nerve (Fig 6(603)), IRIS (Fig 6(604)) and cross sections of IRIS (Fig 6(604)) and ciliary body (Fig 6(605)) etc., are applied on to emissive material (Fig 6(606)) and 3D Eye model (Fig 6(607)) is produced. In similar way the texturing of other embodiments of Type 2 3D models (Fig 2(202)) such as Brain (Fig 2(225)), Skin (Fig 2(227)) and skull (Fig 2(226)) are also created. These accurately created models brings the experience of actual organs to the ophthalmology students and also for the patient’s medical counselling for clinical pathology.

Figure 7 illustrates the anatomy of the Typel 3D models (Fig 2(201 )) like CVS (Fig 7A) and CAS (Fig 7B) respectively. Figure 7A annotates plurality of major parts of Cerebral Venous System (CVS) namely Superior Sagittal Sinus (Fig 7(700)) Transverse Sinus (Fig 7(701)), Superior Petrosal Sinus (Fig 7(702)), Cavernous Sinus (Fig 7(703)), Superior Ophthalmic Veins (Fig 7(704)), Inferior Ophthalmic Veins (Fig 7(705)), Pterygoid Flexus (Fig 7(706)), Inferior Petrosal Sinus (Fig 7(707)), and Internal Jugular Veins (Fig 7(708)) respectively.

Figure 7B (709 to 721 ) annotates plurality of parts of Cerebral Arterial System (CAS), namely, Anterior cerebral artery (Fig 7(709)), Anterior Communicating Artery (Fig 7(710)), Ophthalmic Artery (Fig 7(711)), Middle Cerebral Artery (Fig 7(712)), Internal Carotid Artery (Fig 7(713)), Posterior Communicating Artery (Fig 7(714)), Posterior Cerebral Artery (Fig 7(715)), Superior Cerebellar Artery (Fig 7(716)), Pontine Arteries (Fig 7(717)), Basilar Artery (Fig 7(718)), Anterior Interior Cerebellar Artery (Fig 7(719)), Vertebral Artery(Fig 7(720)), and Anterior Spinal Artery (Fig 7(721)) respectively.

Figure 8 (800 to 806) illustrates the annotation of Type (1) 3D Model of Optic Path Way (800 to 806), the annotations are: - Optic Radiations (Parietal Lobe) (Fig 8(800)), Loop of Meyer (Fig 8(801)), Optic Radiations (Temporal Lobe) (Fig 8(802)), Lateral geniculate body (Fig 8(803)), Optic Tract (Fig 8(804)), Optic Chiasma (Fig 8(805)), and Optic Nerve (Fig 8(806)) respectively. Figure 9 (900 to 925) illustrates the annotated screen shots of Type (1) 3D modelling process output for brain stem Nucleus, The annotations are;- Superior Colliculus (Fig 9(900)), Pretectal Nucleus (Fig 9(901)), Interior Colliculus (Fig 9(902)), Motor Nucleus of Trigeminal Nerve (Fig 9(903)), Cochlear Nucleus (Fig 9(904)), Abducent Nucleus (Fig 9(905)), Spinal Nucleus of Trigeminal (Fig 9(906)), Dorsal Nucleus of Vagus(Fig 9(907)), Hypoglossal Nucleus (Fig 9(908)), Nucleus Ambiguus (Fig 9(909)), Accessory Nucleus (Fig 9(910)) Edinger Westphal Nucleus (Fig 9(911)) Red Nucleus (Fig 9(912)), riMLF (Fig 9(913)), Nucleus of Cajal (Fig 9(914)), Oculo Motor Nucleus (Fig 9(915)),CCN(Fig 9(916)), Trochlear Nerve Nucleus (Fig 9(917)), Mesencephalic Nucleus of Trigeminal (Fig 9(918)), PPRF (Fig 9(919)), MLF (Fig 9(920)), Main sensory Nucleus of Trigeminal (Fig 9(921)), Vestibular Nucleus (Fig 9(922)), Facial Nucleus (Fig 9(923)), Salivatory Nucleus (Fig 9(924)), and Nucleus Solitarius (Fig 9(925)) respectively.

Figure 10 (1000 to 1022) illustrates the plurality of Type 2 3D modelling outputs like Eye ball, Brain, and EOM etc., are illustrated in figures 10A, 10 B and 10 C respectively.

Figure 10 A (1000 to 1008) illustrates the annotations of Extra Ocular Muscles (EOM) of Eye. The annotations are: - Superior Rectus (Fig 10(1000)), Annulus of Zinn (Fig 10(1001)), Lateral Rectus (Fig 10(1002)), Interior oblique Muscles (Fig 10(1003)), Trochlea (Fig 10(1004)), Superior 1 Oblique Muscles (Fig 10(1005)), Levator Pal Pebrae Superioris Muscle (Fig 10(1006)), Medial Rectus (Fig 10(1007)), and Interior Rectus (Fig 10(1008)) respectively.

Figure 10 B (1009 to 1013) illustrates the annotations of 3D model of brain, the annotations are: - Cerebrum (Fig 10(1009)), Pituitary (Fig 10(1010)), Corpus Collosum (Figi 0(1011 )), Ventricles (Fig10(1012)), and Cerebellum (Fig 10(1013) respectively.

Figure 10 C (1014 to 1022) illustrates the annotations of the 3D models of EYE the annotations are, Zonules (Fig10(1014)), Lens (Fig10(1015)), Optic disc (Fig10(1016)), Optic Nerve (Fig10(1017)), Cornea (Fig10(1018)), IRIS (Fig10(1019)), Ciliary Body (Fig10(1020)), Sclera (Fig10(1021)) and Fovea (Fig10(1022)) respectively.

Figure 11 (1100 to 1108) illustrates the Annotations of 3D eye model The Anatomy view on selection from the app displays 3D view of the eye fundus which simultaneously interlinks with the corresponding widgets and cameras and displays the appropriate annotations, enables user to visualize the complete 3D picture of the eye and its internal parts. The 3D display also has the option for switching ON or OFF the vitreous (an aqueous fluid) while viewing. (Fig 11(1100)) as viewed through Eye MG 3D digital tool, Image (a) displays main menu of the “Eye MG 3D” app and contains plurality of tabs like Anatomy, Pathology, Videos and Exit (Figi 1 (1101 )) etc., by clicking the “Anatomy” button (red arrow of image (a)) (Figi 1 (1102)) the eye anatomy (Figi 1 (1103)) can be viewed in detail as illustrated in the Image (b), the user has the option to go to the previous stage by selecting the back arrow (Figi 1 (1104)). By selecting the “Anterior Segment” button (red arrow of image (b)) the annotated anterior segment of the eyeball can be viewed (Figi 1 (1105)). Image (c) displays the options for “Anterior Segment” of the Eye anatomy in the “Eye MG 3D” app. By selecting the “Angle” button (red arrow of Image (c)) (Figi 1 (1105)) the annotated gonioscopic section of the angle (Figi 1 (1106)) can be viewed. Image (d) of the Anterior Segment in the “Eye MG 3D” app illustrates the view of “Angle”. (Figi 1 (1107)) in the “Eye MG 3D” app. By selecting the “Trabecular Meshwork” button (red arrow of Image (d)) user can view the electron microscopic structure of the angle. The image (e) illustrates the detailed annotations of “Trabecular meshwork” (Figi 1 (1108)) in the “Eye MG 3D” application.

Figure 12 (1200 to 1213) illustrates the Annotations of 3D eye model (Fig 12(1200)) as viewed through Eye MG 3D digital tool, Image (a) illustrates main menu of the “Eye MG 3D” app, which Contains tabs Anatomy, Pathology, Videos and Exit (Fig12(1201)) respectively by selecting the “Pathology” button (red arrow of image (a)) the plurality of ocular pathologies (Fig12(1203)) of the eye fundus (Fig12(1204)) can be viewed in detail as illustrated in the Image (b), the user has the option to go to the previous stage by selecting the back arrow (Fig12(1205)). On selecting the “Anterior Segment” button (Fig12(1202)) the pathologies of anterior segment of the eyeball are displayed. By selecting the “Flecked retina” button (red arrow) (Fig12(1203)) from the drop-down as illustrated in Image (c) the pathology condition of eye fundus for “Flecked retina” (Fig12(1207)) is displayed in colour mode when “Color” button (red arrow) (Fig12(1208)) is being selected. Image (d) illustrates the pathology condition of eye fundus for “Flecked Retina” (Fig12(1209)) of the eye fundus (Fig12(1210)) in Infrared model which is obtained by selecting the “IR” button (red arrow) (Figi 2(1212)). To view the image with vitreous fluid effect the “Vitreous” button must be selected ON (Figi 2(1211 )). Image (e) illustrates the pathology condition of eye fundus for “Flecked Retina” (Fig12(1209)) in “Auto Fluorescence” mode which is displayed by selecting the “AF” button (red arrow) (Fig12(1213)). Similarly, user can view plurality of pathological conditions (Fig 2(248)) of the eye fundus such as i) Neo Vascularization; ii) Disc Coloboma; iii) Retinitis Pigmentosa; iv) Myelinated Nerve Fiber; v) Splinter Hemorrhage, vi) RNFL Defect etc., The pathological conditions mentioned above are not exhaustive as continuously new conditions are being regularly created and added and it’s ever evolving.

Thus, the description clearly specifies the invention and the manner in which it is to be performed.

TECHNICAL ADVANTAGES AND ECONOMIC

SIGNIFICANCE OF THE INVENTION

The instant invention provides a creative digital system and 3D tool for training and medical counseling in the field of Ophthalmology, in a much more factual, fun, and easy to access way. The eye being a complex structure, wherein human eye dissection for students is not being practical and the young students and novice surgeons have to mentally visualize these concepts pertaining to ophthalmology in a theoretical framework, the invention thus provides smart teaching methods to help students of ophthalmology for proper understanding of the anatomy and pathology of eye in the absence of dissection or any other objectified training aids to make the study more interested and fun filled like a gaming application in a most cost effective manner.

Present invention is aimed at development of a digital system and 3D tool for training and medical counselling in ophthalmology and as the capability to integrate with virtual simulators for training of specialized physicians, mainly in surgical education.

While the present invention has been delineated by description of a preferred embodiment and the preferred embodiment has been described in considerable details with reference to the drawings, it is not contemplated to confine or in any way restrict the scope of the appended claims and their legal equivalence to such realization. Additional applications and adaptations will readily appear to those skilled in the art. The invention in its broader aspects is therefore not bound to the specific details that are representative of apparatus and method, and illustrative features shown and described herein. The inventions being thus described, it will be obvious that the same may be varied in many ways. Such variations are not to be regarded as a departure from the scope of the inventions and all such modifications are intended to be included within the scope of the following claims and equivalents thereof.

Claims

We Claim,

1 . Digital system and 3D tool for training and medical counselling in Ophthalmology comprising of

2D image source for conversion to 3D models (Fig 1 (100), the process comprising of 2D image (Fig 5 (501)) from fundus scanner (Fig 5 (500)) or other sources being mapped (Fig 5 (505)) with UV layout (Fig 5 (503)) of the modelling platform by super imposition (Fig 5 (504)) of the corresponding texture (Fig 5 (509)) on to the plain 3D model (Fig 5 (508)) to create textured 3D eye fundus (Fig 5 (510)) and plurality of other models;

3D modelling process (Figi (106) & (Fig 2 (200)) to create 3D models (Figi (107), through plurality of modelling process comprising Type (1) process (Fig 2 (201)) for smaller blood vessels, comprises of drawing (Fig 2 (203)) of 3D models in the software tool, carryout extrusion along the path of the drawing (Fig 2 (204)) in a circular form to match the natural shape and size of the desired model and adjust the thickness (Fig 2 (205)) to create colorless model (Fig 2 (206)) of the desired structure, further application of the new material to this model (Fig 2 (208)) and assign appropriate color (Fig 2 (207)) to the model and complete the type 1 modelling process (Fig 2 (210)) and Type (2) modelling process (Fig 2 (202)) for whole organs, comprises of selection of basic shape of the organ (Fig 2 (216)) from the software application (Fig 2 (217)), molding it into colorless models (Fig 2 (206)) to super impose and map with UV layout (Fig 2 (219)) of the appropriate texture (Fig 2 (220)) and complete the model for eye (Fig 2 (223)) and plurality of other models; Integration of 3D models with digital (Fig 2 (229)) or Analog tools (Fig 2

(230)) and devices (Fig 2 (228)), to visualize anatomy (Fig 2 (236)) and pathology (Fig 2 (237)) of the eye fundus and other organs; development and integration of 3D image viewing application (Figi (110) and (Fig 2 (228)) to harvest plurality of utilities such as, study of anatomy (Figi (111) and (Fig 2 (236)), study of pathology (Figi (112) and (Fig 2 (237)), conduction of surgical simulations (Fig 1 (114) and (Fig 2 (251 )), Integration to patient’s medical records (Figi (115) and (Fig 2 (250)) to achieve healthcare automation and; alternately integration of 3D models with analog process (Figi (109) and (Fig 2 (230)) for 3D printing (Figi (116) and (Fig 2 (235)) to create physical models for study of ophthalmology. Digital system and 3D tool for training and medical counselling in Ophthalmology as claimed in claim (1 ), the Type (1) modelling output (Fig 2 (210)) comprises of plurality of smaller blood vessels comprising of Cerebral Venus System (CVS) (((Fig 2 (212)) and (Fig 7A))), Cerebral Artery System (CAS) (((Fig 2 (211)) and (Fig 7B))), Cerebral Nervous System (CNS) (Fig 2 (213)), Brain Stem Nucleus (BSN) (((Fig 2 (214)) and (Fig 9))), Optic Path way (((Fig 2 (216)) and (Fig 8))) and like. Digital system and 3D tool for training and medical counselling in Ophthalmology as claimed in claim (1 ), the Type (2) 3D modelling output (Fig 2 (223)) comprises of plurality of whole organs comprising Extra Ocular Muscles (EOM) of Eye (((Fig 2 (224)) and (Fig 10A))), Brain (((Fig 2 (225)) and (Fig 10B))), Eye (Fig 10C ), Skull (Fig 2 (226)), Skin (Fig 2 (227)) and like. Digital system and 3D tool for training and medical counselling in Ophthalmology as claimed in claim (1 ), the digital tool (Fig 2 (229)) comprises plurality of devices comprising personal computers (PCs), Android, IOS, Hollo Lens and like and Analog tools (Fig 2 (230)) for 3D printing to produce hand held physical model (Fig 2 (235)). Digital system and 3D tool for training and medical counselling in Ophthalmology as claimed in claim (1 ), the 3D viewing comprises of plurality of digital applications (Fig 2 (229)) comprising Eye MG 3 D (Fig 2 (231)) to operate on PCs and Android, Eye MG AR (Fig 2 (232)) for viewing pathological conditions of organs with augmented technology enablement, Eye MG Max (Fig 2 (233)) to operate on iPhone, Eye MG Holo (Fig 2 (234)) developed for HoloLens2 device with cutting- edge technology having features including spatial recognition, hand recognition, iris tracking and like. Digital system and 3D tool for training and medical counselling in Ophthalmology as claimed in claim (1 ), the system is integrated with Eye MG Holo (Fig 2 (234)) for the plurality of surgical simulation activities (Fig 2(251)) comprising of Trabeculectomy, Foreign body removal, Stent Placement, Cataract Surgery, Intra Ocular Foreign Body Removal, Retinal Detachment, Vitrectomy, Peripheral Iridectomy and like. Digital system and 3D tool for training and medical counselling in Ophthalmology as claimed in claim (1 ), the system is integrated with Eye MG Holo for digitizing the health care process, comprising of examination of patient and record necessary scans and screening details into the patient data base, which Eye MG Holo (Fig 2((234)) converts into 3D models of the organs like Eye, for further analysis and study of the patients. Digital system and 3D tool for training and medical counselling in Ophthalmology as claimed in claim (1 ), Integration of 3D models with digital (Fig 2 (229)) tools and devices (Fig 2 (228)), comprises of plurality of visualizing options for anatomy (Fig 2 (236)) and pathology (Fig 2 (237)) of the eye fundus and other organs, the process of visualization comprises of tab selection in the app to interlink with the corresponding widgets and cameras and enable user to visualize the appropriate annotations (Figi 1 (1103)) and the complete 3D picture of the eye and its internal parts and alternately on selection the user can view plurality of ocular pathologies (Fig12(1203)) of the eye fundus Digital system and 3D tool for training and medical counselling in Ophthalmology as claimed in claim (1 ), the pathology (Fig 2 (237)) conditions of the eye fundus include plurality of pathologies comprising (Fig 2 (248)) Neo Vascularization, Disc Coloboma, Retinitis Pigmentosa, Myelinated Nerve Fiber, Splinter Hemorrhage, RNFL Defect and like.