WO2021244074A1 - 基于虚拟现实技术的医学影像显示方法及*** - Google Patents

基于虚拟现实技术的医学影像显示方法及*** Download PDF

Info

Publication number
WO2021244074A1
WO2021244074A1 PCT/CN2021/077839 CN2021077839W WO2021244074A1 WO 2021244074 A1 WO2021244074 A1 WO 2021244074A1 CN 2021077839 W CN2021077839 W CN 2021077839W WO 2021244074 A1 WO2021244074 A1 WO 2021244074A1
Authority
WO
WIPO (PCT)
Prior art keywords
geometric model
medical image
virtual reality
session
dimensional geometric
Prior art date
Application number
PCT/CN2021/077839
Other languages
English (en)
French (fr)
Inventor
刘非
徐显辉
Original Assignee
上海昕健医疗技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海昕健医疗技术有限公司 filed Critical 上海昕健医疗技术有限公司
Publication of WO2021244074A1 publication Critical patent/WO2021244074A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS

Definitions

  • the invention relates to the field of virtual reality technology, in particular to a medical image display method, a medical image processing system and a multi-person collaborative interaction method based on virtual reality technology.
  • VR Virtual Reality
  • the existing virtual reality technology in the medical field mainly uses X-ray computer tomography (CT), nuclear magnetic resonance (MRI) and other medical imaging technologies to scan the living body or corpse in an all-round way to obtain human tissue Organ information, and then directly build the scanned human body model through 3D reconstruction, and use virtual reality technology and equipment to present the virtual model in the scene, so that the user can see a 3D restoration of the scanned human body structure in the scene
  • the purpose of the present invention is to provide a medical image processing method, device and corresponding equipment based on virtual reality technology, which can realize three-dimensional geometric model reconstruction, virtual reality scene display and virtual reality scene synchronization.
  • the present invention provides a medical image processing system based on virtual reality technology, which includes a control unit, a data processing unit, a display unit, an interaction unit, and a transmission unit that are communicatively connected to each other.
  • the control unit instructs the display unit to
  • the three-dimensional geometric model data is displayed in the VR device;
  • the interactive unit transforms the three-dimensional geometric model in the VR device by acquiring interactive commands in real time;
  • the transmission unit realizes interactive synchronous transformation by transmitting the transformation parameters of the three-dimensional geometric model.
  • the data processing unit includes an acquisition unit, a segmentation unit, and a reconstruction unit, the acquisition unit is used to acquire medical image data; the segmentation unit is used to segment the target area in the medical image data to obtain a segmentation result; The reconstruction unit is used to perform three-dimensional reconstruction on the segmentation result to generate three-dimensional geometric model data.
  • the display unit includes a first subunit for displaying medical images, a second subunit for displaying a three-dimensional model, and a third subunit for displaying the VR device.
  • the present invention provides a medical image display method based on virtual reality technology, which is characterized in that it includes the following steps:
  • the VR device synchronously transforms and operates the three-dimensional geometric model according to the transform parameters and displays it.
  • the medical image display method further includes the following steps: S4. Multiple VR devices are connected at the same time, the synchronized reconstruction grid data is downloaded, the collaborative interactive session is entered, and the three-dimensional geometric model is displayed simultaneously.
  • the medical image display method further includes the following steps: S5. Synchronously interact with the three-dimensional geometric model, transform the three-dimensional geometric model through the current VR device, and send the processed parameters to other VR devices simultaneously. Transform the corresponding three-dimensional geometric model and display it.
  • step S5 includes step S51.
  • the 3D geometric model is locked, so that other VR devices cannot operate on the 3D geometric model. ;
  • the 3D geometric model is unlocked and the unlocking information is transmitted to other VR devices.
  • step S5 includes step S53.
  • the newly added VR device downloads and synchronizes the reconstructed grid data through connection, enters the session, and displays the three-dimensional geometric model synchronously.
  • step S5 includes step S54.
  • the VR device exits the session at any time by disconnecting, and after all VR devices exit the session, the session is closed.
  • S0 Obtain medical images and data, and perform preprocessing; perform three-dimensional reconstruction to obtain three-dimensional geometric model data.
  • the present invention provides a method for multi-person collaborative interaction based on virtual reality technology, which includes the following steps:
  • step S4 includes step S41.
  • the three-dimensional geometric model is locked, so that other VR devices cannot operate on the three-dimensional geometric model. ;
  • the 3D geometric model is unlocked and the unlocking information is transmitted to other VR devices.
  • step S4 includes step S42.
  • the newly added VR device downloads and synchronizes the reconstruction grid data through connection, enters the session, and displays the three-dimensional geometric model synchronously; the VR device disconnects the connection When you exit the session at any time, after all VR devices exit the session, the session is closed.
  • the embodiment of the present invention performs three-dimensional geometric reconstruction after acquiring medical image data, and then starts multiple VR devices, displays the medical image data and three-dimensional geometric model in the multiple VR devices, and compares the medical image data and the reconstruction model through user interaction commands
  • the data can be transformed to observe the medical image data and reconstruct the model data in the virtual reality scene, so that the display result is more three-dimensional and lifelike, and the condition of the disease can be observed and diagnosed more accurately.
  • the parameter transmission used in this method is different from the traditional model data transmission, can reduce the amount of transmitted data, eliminate the delay, can achieve real-time and instant effects, and make the VR display effect continuous, smooth, and without frustration.
  • the display content of multiple VR devices can be synchronized, and multiple people can observe a case at the same time, which is helpful for discussion and communication before and after surgery.
  • FIG. 1 is a schematic diagram of the connection of the medical image processing system based on virtual reality technology with the VR equipment and the server of the present invention.
  • Fig. 2 is a flowchart of the medical image display method based on virtual reality technology of the present invention.
  • Fig. 3 is a schematic diagram of the medical image processing system based on virtual reality technology of the present invention.
  • the medical imaging system based on virtual reality technology of the present invention is used to operate, display, and watch in medical medical scenes in cooperation with the virtual reality system.
  • the medical image processing device may be integrated in a network device, which may be a server 1 or a terminal 2, and the terminal 2 may include a notebook computer, a personal computer, and other devices.
  • the virtual reality system of the present invention includes a VR device 3, a locator, and a handle.
  • the VR device 3 is its main display device.
  • the locator is used to assist positioning during the interaction process, and the handle is used for user interaction. Referring to FIG. 1, data can be transmitted to other terminals 2 through the server 1 and displayed by the VR device 3.
  • a VR device is a way of closing people's external visual and auditory senses, and guiding users to have a feeling of being in a virtual environment.
  • the principle is that the left and right screens display the images of the left and right eyes respectively, and the human eyes produce a three-dimensional effect in their minds after obtaining this different information.
  • the VR device 3 of the present invention may be a head-mounted electronic device such as a VR display helmet and VR glasses.
  • the VR device 3 may also be equipped with detection equipment such as gloves, bracelets, patches, etc., which are not used in the present invention. limited.
  • the medical imaging system of the present invention based on virtual reality technology is also used in conjunction with AR.
  • the only difference is whether the displayed content is all artificially virtual or superimposed on a real scene. This problem can be easily understood by those skilled in the art without having to entangle any specific concepts.
  • the aforementioned AR also includes MR, and currently there is no strict distinction between the two technically.
  • the medical image processing system based on virtual reality technology includes a control unit, a data processing unit, a display unit, an interaction unit, and a transmission unit that are communicatively connected to each other, and the control unit instructs the display unit to display three-dimensional geometric model data in a VR device;
  • the interactive unit transforms the three-dimensional geometric model in the VR device by acquiring interactive commands in real time; the transmission unit realizes interactive synchronous transformation by transmitting the transform parameters of the three-dimensional geometric model.
  • the data processing unit further includes an acquisition unit, a segmentation unit, and a reconstruction unit.
  • the acquisition unit is used for acquiring medical image data, and grouping and filtering the read medical image sequences.
  • the segmentation unit is used to segment the target area in the medical image data to obtain a segmentation result.
  • the reconstruction unit is used to perform three-dimensional reconstruction on the segmentation result to generate three-dimensional geometric model data.
  • the segmentation method of the above-mentioned segmentation unit includes a manual segmentation method, a semi-automatic segmentation method, and other processing methods for segmentation results.
  • the segmentation result is displayed in the corresponding position of the medical image, marked with different colors.
  • Manual segmentation methods include single-layer image editing, multi-layer image editing, lasso editing and other manual segmentation methods.
  • Semi-automatic segmentation includes region growth segmentation, watershed segmentation, level set segmentation and graph cut segmentation methods.
  • Other processing methods of segmentation results include morphological operations, hole filling, smoothing, and Boolean operations.
  • the display unit further includes a first subunit for displaying medical images, a second subunit for displaying a three-dimensional model, and a third subunit for displaying the VR device.
  • the first subunit displays the medical image and the segmentation result, and uses a specific color to mark the segmentation result at the position corresponding to the medical image.
  • the second subunit displays the three-dimensional geometric model, different three-dimensional geometric models are marked with specific colors.
  • the transformation parameters transmitted by the transmission unit include translation transformation parameters, rotation transformation parameters, scaling transformation parameters, cutting transformation parameters, and reset transformation parameters.
  • the parameter transmission method adopted by the medical imaging system based on virtual reality technology of the present invention is different from the traditional model data transmission method. It can reduce the amount of data transmitted, eliminate delay, achieve real-time effects, and make the VR display effect continuous, smooth, and smooth. No frustration. Especially in the state of being connected to multiple VR devices, the display content of multiple VR devices can be synchronized.
  • the above medical image processing system is used to implement a medical image display method based on virtual reality technology, and the medical image display method includes the following steps:
  • the VR device synchronously transforms and operates the three-dimensional geometric model according to the transform parameters and displays it.
  • step S0 medical image acquisition may be performed by CT, MRI or ultrasound imaging equipment, and then image preprocessing may be performed.
  • the target area of the medical image data acquired in step S0 is segmented.
  • the segmentation method is not limited to one, and can include purely manual segmentation methods, threshold segmentation, region growing, level set, watershed and artificial intelligence segmentation methods, and can also include multiple segmentation methods combined in different orders Formed processing method.
  • the target area can be human tissues and organs in the medical image data, such as bones, heart, liver, lungs, etc., or can be any irregularly shaped area desired by the user.
  • step S0 the segmentation result is regenerated as three-dimensional geometric model data.
  • the three-dimensional geometric model data can be composed of triangular faces, or can be composed of tetragonal faces, tetrahedrons, hexahedrons, and the like.
  • step S1 sharing the 3D geometric model data is realized by the transmission unit of the medical image processing system.
  • the 3D geometric model data is automatically loaded, rendered and displayed in the VR device, and multiple 3D geometries can be displayed in the VR device Model.
  • step S2 the user interacts with the three-dimensional geometric model through the handle.
  • the user can select a certain three-dimensional geometric model to realize the transformation of the selected model.
  • the transformation includes: translation, rotation, zoom and reset, and can also choose to hide the selected three-dimensional geometry Model.
  • the translation transformation is to calculate the translation amount in the three directions of the X-axis, Y-axis and Z-axis of the transformed model during the translation process.
  • the rotation transformation is to calculate the rotation matrix of the transformation model through the quadtree during the rotation process.
  • the scaling transformation is to calculate the scaling ratio of the transformation model during the scaling process.
  • Reset transformation is to restore the display state of all models to the initial display state.
  • step S3 the parameters of the current VR device transmission transformation are transmitted to other VR devices through the medical image processing system, including translation amount, transformation matrix, zoom ratio and rotation matrix, and other VR devices are transformed at the same time to realize multiple VR devices Display content synchronization.
  • This method can reduce the amount of transmitted data, eliminate the delay, and achieve real-time effects, so that the VR display effect is continuous, smooth, and without frustration.
  • the present invention provides a multi-person collaborative interaction method based on virtual reality technology.
  • the multi-person collaborative interaction method includes the following steps:
  • each VR device needs to verify the project file information of the currently active user to the server.
  • the verification information includes the session ID number, patient name, scan time and other information included in the case.
  • Step S4 further includes step S41.
  • the 3D geometric model is locked, so that other VR devices cannot operate on the 3D geometric model; the current VR device After the 3D geometric model is unchecked, the 3D geometric model is unlocked and the unlock information is transmitted to other VR devices.
  • Step S4 further includes step S42.
  • the newly added VR device downloads the synchronized reconstruction grid data through connection, enters the session, and displays the 3D geometric model simultaneously; the VR device exits the session at any time by disconnecting, After all VR devices exit the session, the session is closed.
  • the user corresponding to each VR device can pick and select the 3D geometric model.
  • the 3D geometric model informs other VR devices that the model is selected and locked through the server.
  • the locked 3D geometric model can only be operated by the user until the user deselects the model, and the server notifies other users that the locked model is unlocked and can be picked and selected.
  • the 3D geometric model can be rotated, translated, zoomed, and cut through the handle.
  • the parameters of each processing will be sent to other VR devices through the server synchronously.
  • Other VR devices Transform and display the corresponding three-dimensional geometric model.
  • each user can exit the session at any time, or some users can join the session at any time, until all users exit the session, the server will close the session.
  • This scene is commonly used in consultations conducted by multiple doctors on the same case.
  • Medical image data and reconstruction model data can be observed in a virtual reality scene, making the displayed results more realistic, and more accurate observation and diagnosis of the condition.
  • Multiple VR devices are connected and interact synchronously, allowing multiple people to observe a case at the same time, which is helpful for discussion and communication before and after surgery.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Geometry (AREA)
  • Computer Graphics (AREA)
  • Radiology & Medical Imaging (AREA)
  • Epidemiology (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Primary Health Care (AREA)
  • Public Health (AREA)
  • Processing Or Creating Images (AREA)

Abstract

本发明提供一种基于虚拟现实技术的医学影像显示方法、医学影像处理***及多人协同交互方法。医学影像显示方法包括如下步骤：共享三维几何模型数据，使VR设备自动加载并在其中渲染显示三维几何模型；根据用户交互命令，对VR设备显示场景中的三维几何模型进行变换，得到变换参数；VR设备根据变换参数，同步变换操作三维几何模型并显示。医学影像处理***包括相互通讯连接的控制单元、数据处理单元、显示单元、交互单元和传输单元。

Description

基于虚拟现实技术的医学影像显示方法及***
本申请要求了申请日为2020年6月3日，申请号为202010494407.X，发明名称为“基于虚拟现实技术的医学影像显示方法及***”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及虚拟现实技术领域，尤其涉及一种基于虚拟现实技术的医学影像显示方法、医学影像处理***及多人协同交互方法。
背景技术
人机交互技术是目前用户界面技术中发展迅速的领域之一，现已获得大量研究成果。虚拟现实作为一门新兴人机交互技术，具有更广阔的应用前景。虚拟现实(Virtual Reality，简称VR)技术是80年代随着计算机图形仿真技术的深入研究而发展起来的一个新的研究领域，其可以模拟产生一个多源信息融合、交互式的三维空间的虚拟环境，给体验者提供身临其境的模拟真实体验感。现有的虚拟现实技术在医学领域中的应用，主要是利用X射线计算机断层成像(CT)、核磁共振(MRI)等医学影像技术，对人体活体或尸体进行全方位的扫描，从而获取人体组织器官信息，再通过三维重建直接建立扫描的人体模型，应用虚拟现实技术和设备将虚拟模型呈现在场景之中，使用户在场景中看到一个三维还原的扫描人体结构
虽然目前关于视觉合成的研究较多，但用于医疗医学领域的产品相对较少，且目前大多虚拟现实***均只有单一的视觉立体或触觉立体，无法实现视觉与触觉的同步匹配，代入感不强。尤其是可以由多人参与的虚拟场景下，如何可以独立地进行体验，又可以与其他体验者之间互不影响，甚至交互协作，是现在亟需解决的技术难题。
因此，确有必要提供一种基于虚拟现实技术的医学影像显示方法及其使用方法，以克服现有技术中存在的缺陷。
发明内容
本发明的目的在于提供一种基于虚拟现实技术的医学影像处理方法、装置及相应设备，能够实现三维几何模型重建，虚拟现实场景显示和虚拟现实场景同步。
为实现上述目的，本发明提供了一种基于虚拟现实技术的医学影像处理***，包括相互通讯连接的控制单元、数据处理单元、显示单元、交互单元和传输单元，所述控制单元指令 显示单元在VR设备中显示三维几何模型数据；所述交互单元通过实时获取交互命令，对VR设备中的三维几何模型进行变换；所述传输单元通过传输三维几何模型的变换参数，实现交互同步变换。
进一步地，所述数据处理单元包括采集单元、分割单元和重建单元，所述采集单元用于获取医学影像数据；所述分割单元用于对医学图像数据中的目标区域进行分割，得到分割结果；所述重建单元用于对分割结果进行三维重建，生成三维几何模型数据。
进一步地，所述显示单元包括用于显示医学图像的第一子单元、用于显示三维模型的第二子单元和用于VR设备显示的第三子单元。
为实现上述目的，本发明提供了一种基于虚拟现实技术的医学影像显示方法，其特征在于：包括如下步骤：
S1.共享三维几何模型数据，使得VR设备自动加载并在其中渲染显示三维几何模型；
S2.根据用户交互命令，对VR设备显示场景中的三维几何模型进行变换，得到变换参数；
S3.VR设备根据变换参数，同步变换操作三维几何模型并进行显示。
进一步地，所述医学影像显示方法还包括如下步骤：S4.多个VR设备同时连接，下载同步重建网格数据，进入协同交互会话，并同步显示三维几何模型。
进一步地，所述医学影像显示方法还包括如下步骤：S5.对三维几何模型进行同步交互，通过当前VR设备对三维几何模型进行变换处理，处理后的参数同步发送至其他VR设备，其他VR设备将对应的三维几何模型变换处理后进行显示。
进一步地，在步骤S5中，包括步骤S51.在协同交互会话过程中，当前VR设备对三维几何模型进行拾取选中后，该三维几何模型被锁定，使得其他VR设备不能对该三维几何模型进行操作；当前VR设备对三维几何模型取消选中后，该三维几何模型解除锁定，并将解锁信息传送至其他VR设备。
进一步地，在步骤S5中，包括步骤S53.在协同交互会话过程中，新增加的VR设备通过连接，下载同步重建网格数据，进入会话，并同步显示三维几何模型。
进一步地，在步骤S5中，包括步骤S54.在协同交互会话过程中，VR设备通过断开连接而随时退出会话，所有VR设备退出会话后，该会话关闭。
进一步地，还包括如下步骤，S0.获取医学影像及数据，并进行预处理；进行三维重建，得到三维几何模型数据。
为实现上述目的，本发明提供了一种基于虚拟现实技术的多人协同交互方法，包括以下步骤：
S1.启动服务器，并连接多个VR设备；
S2.启动各个VR设备向服务器验证信息；
S3.下载同步重建网格数据，开始协同交互会话；
S4.对三维几何模型进行同步交互，通过当前VR设备对三维几何模型进行变换处理，处理的参数通过服务器同步发给其他VR设备，其他VR设备会将对应的三维几何模型进行变换处理后进行显示。
进一步地，在步骤S4中，包括步骤S41.在协同交互会话过程中，当前VR设备对三维几何模型进行拾取选中后，该三维几何模型被锁定，使得其他VR设备不能对该三维几何模型进行操作；当前VR设备对三维几何模型取消选中后，该三维几何模型解除锁定，并将解锁信息传送至其他VR设备。
进一步地，在步骤S4中，包括步骤S42.在协同交互会话过程中，新增加的VR设备通过连接，下载同步重建网格数据，进入会话，并同步显示三维几何模型；VR设备通过断开连接而随时退出会话，所有VR设备退出会话后，该会话关闭。
本发明的实施例在获取医学影像数据后，进行三维几何重建，然后启动多个VR设备，在多个VR设备中显示医学影像数据和三维几何模型，通过用户交互命令对医学图像数据和重建模型数据进行变换，可以在虚拟现实场景中观察医学影像数据和重建模型数据，使显示结果更加立体逼真，可以更准确的观察和诊断病情。本方法采用的参数传输区别于传统的模型数据传输，可以减少传输的数据量，消除延时，能够达到实时、即时的效果，并使得VR显示效果连续、平滑、无顿挫感。尤其是在与多个VR设备连接的状态下，能够实现多个VR设备显示内容的同步，多人同时观察一个病例，有助于术前术后的讨论交流。
附图说明
图1为本发明的基于虚拟现实技术的医学影像处理***与VR设备和服务器连接的示意图。
图2为本发明的基于虚拟现实技术的医学影像显示方法的流程图。
图3为本发明的基于虚拟现实技术的医学影像处理***的示意图。
具体实施方式
为使本发明的上述目的、特征和优点能够更为明显易懂，下面结合本发明实施例中的附图，对本发明实施例中的技术方案进行清除、完整的描述，显然，所描述的实施例仅仅是本发明的一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域的技术人员在没有做出创造性的劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
请结合图1至图3所示，本发明基于虚拟现实技术的医学影像***，用于在医疗医学场景配合虚拟现实***进行操作、显示、观看。所述医学影像处理装置可以集成在网络设备中，该网络设备可以是服务器1，也可以是终端2等设备；其中终端2可以包括笔记本电脑和个人计算机等设备。
本发明的虚拟现实***包括VR设备3、***、手柄，其中VR设备3是其主要显示装置，***用来在交互过程中辅助定位，手柄用于用户的交互。参照图1所示，通过服务器1可将数据传输至其他终端2，并通过VR设备3进行显示。
VR设备是一种将人的外界的视觉、听觉封闭，引导用户产生一种身在虚拟环境中感觉。其原理是左右屏幕分别显示左右眼的图像，人眼获取这种有差异的信息后在脑海中产生立体感。实际应用中，本发明的VR设备3可以是VR显示头盔、VR眼镜等头戴式电子设备，其中，VR设备3还可以配备手套、手环、贴片等检测设备，在本发明中不做限定。
理论上，本发明基于虚拟现实技术的医学影像***也与AR配合统使用，区别仅在于显示的内容是全部由人工虚拟的，还是在现实场景上叠加的。这个问题，本领域技术人员很容易理解而无需纠结任何具体的概念。当然，上述AR也包括MR，目前在技术上两者并没有严格的区分。
所述基于虚拟现实技术的医学影像处理***包括相互通讯连接的控制单元、数据处理单元、显示单元、交互单元和传输单元，所述控制单元指令显示单元在VR设备中显示三维几何模型数据；所述交互单元通过实时获取交互命令，对VR设备中的三维几何模型进行变换；所述传输单元通过传输三维几何模型的变换参数，实现交互同步变换。
所述数据处理单元进一步包括采集单元、分割单元和重建单元。所述采集单元用于获取医学影像数据，并对读取的医学影像序列进行分组筛选。所述分割单元用于对医学图像数据中的目标区域进行分割，得到分割结果。所述重建单元用于对分割结果进行三维重建，生成三维几何模型数据。
在本发明中，上述分割单元的分割方法包括手动分割方法、半自动分割方法和对分割结果的其他处理方法。分割结果显示在医学影像对应的位置，用不同的颜色标记。手动分割方法包括单层图像编辑，多层图像编辑，套索编辑等手动分割方法。半自动分割包括区域生长分割、分水岭分割、水平集分割和图割分割方法。分割结果的其他处理方法包括形态学操作、空洞填充、平滑和布尔运算等处理方法。
所述显示单元进一步包括用于显示医学图像的第一子单元、用于显示三维模型的第二子单元和用于VR设备显示的第三子单元。
所述第一子单元显示医学影像和分割结果，用特定的颜色在医学影像对应得位置标记分割结果。所述第二子单元显示三维几何模型时，用特定的颜色标记不同的三维几何模型。
所述传输单元传输的变换参数包括平移变换参数、旋转变换参数、缩放变换参数、切割变换参数和重置变换参数。
本发明基于虚拟现实技术的医学影像***采用的参数传输方法，区别于传统的模型数据传输方法，可以减少传输的数据量，消除延时，达到实时的效果，并使得VR显示效果连续、平滑、无顿挫感。尤其是在与多个VR设备连接的状态下，能够实现多个VR设备显示内容的同步。
上述医学影像处理***用于实现基于虚拟现实技术的医学影像显示方法，该医学影像显示方法包括如下步骤：
S0.获取医学影像及数据，并进行预处理；进行三维重建，得到三维几何模型数据；
S1.共享三维几何模型数据，使得VR设备自动加载并在其中渲染显示三维几何模型；
S2.根据用户交互命令，对VR设备显示场景中的三维几何模型进行变换，得到变换参数；
S3.VR设备根据变换参数，同步变换操作三维几何模型并进行显示。
在步骤S0中，可通过CT、MRI或者超声成像设备进行医学影像采集，然后进行影像预处理。
进一步，对步骤S0中获取的医学影像数据的目标区域进行分割。
在本发明中，分割方法不局限于一种，可以包括纯手工分割方法，也可以包括阈值分割、区域生长、水平集、分水岭和人工智能分割方法，还可以包括多种分割方法以不同顺序组合形成的处理方法。
目标区域可以是医学影像数据中的人体组织器官，例如骨骼、心脏、肝脏、肺等，也可以是用户想要得到的任意不规则形状区域。
在步骤S0中，将分割结果重新生成为三维几何模型数据。该三维几何模型数据可以由三角面片组成，亦可以由四角面片、四面体、六面体等组成。
在步骤S1中，共享三维几何模型数据通过医学影像处理***的传输单元实现，当启动VR设备时，自动加载三维几何模型数据，在VR设备中渲染显示，在VR设备中可以显示多个三维几何模型。
在步骤S2中，用户通过手柄与三维几何模型进行交互，用户可以选取某一个三维几何模型，实现对选取模型的变换，变换包括：平移、旋转、缩放和复位，也可以选择隐藏选取 的三维几何模型。
在本发明中，平移变换是在平移过程中计算变换模型X轴、Y轴和Z轴三个方向上的平移量。旋转变换是在旋转过程中通过四叉树计算变换模型的旋转矩阵。缩放变换是在缩放过程中计算变换模型的缩放比例。重置变换是将所有模型的显示状态恢复到初始显示状态。
在步骤S3中，通过医学影像处理***将当前VR设备传输变换的参数传输给其他VR设备，包括平移量，变换矩阵，缩放比例和旋转矩阵，同时对其他VR设备进行变换，实现多个VR设备显示内容的同步。这种方法可以减少传输的数据量，消除了延时，能够达到实时的效果，以使得VR显示效果连续、平滑、无顿挫感。
本发明为了实现了多个VR设备之间模型变换的实时同步，提供了一种基于虚拟现实技术的多人协同交互方法。所述多人协同交互方法包括以下步骤：
S1.启动服务器，并连接多个VR设备；
S2.启动各个VR设备向服务器验证信息；
S3.下载同步重建网格数据，开始协同交互会话；
S4.对三维几何模型进行同步交互，通过当前VR设备对三维几何模型进行变换处理，处理的参数通过服务器同步发给其他VR设备，其他VR设备会将对应的三维几何模型进行变换处理后进行显示。
在步骤S2中，各个VR设备需要向服务器验证当前启动用户的项目文件信息，验证信息包括病例中包括的会话ID号、病人姓名、扫描时间等信息。
步骤S4进一步包括步骤S41.在协同交互会话过程中，当前VR设备对三维几何模型进行拾取选中后，该三维几何模型被锁定，使得其他VR设备不能对该三维几何模型进行操作；当前VR设备对三维几何模型取消选中后，该三维几何模型解除锁定，并将解锁信息传送至其他VR设备。
步骤S4进一步包括步骤S42.在协同交互会话过程中，新增加的VR设备通过连接，下载同步重建网格数据，进入会话，并同步显示三维几何模型；VR设备通过断开连接而随时退出会话，所有VR设备退出会话后，该会话关闭。
上述步骤S41和S42并无先后顺序，数字仅做区分标识。
具体来说，在交互会话过程中，每个VR设备对应的用户都可以对三维几何模型进行拾取选中，三维几何模型被选中之后，该三维几何模型通过服务器通知其他VR设备该模型被选中锁定，锁定的三维几何模型只能被该用户操作，直到该用户取消选中模型，服务器通知其他用户，被锁定模型解除锁定，可以被拾取选中。
在交互会话过程中，一个用户锁定三维几何模型后，可以对通过手柄对三维几何模型进行旋转、平移、缩放、切割处理，每次处理的参数会通过服务器同步发给其他VR设备，其他VR设备将对应的三维几何模型进行变换处理并进行显示。
在交互会话过程中，每个用户可以随时退出会话，也可以有用户随时加入会话，直到所有的用户退出会话后，服务器会关闭该会话。
该场景常见应用于多个医生对同一病例进行的会诊，可以在虚拟现实场景中观察医学影像数据和重建模型数据，使显示结果更加立体逼真，并且可以更准确的观察和诊断病情。多个VR设备连接并同步交互，使得多人同时观察一个病例，有助于术前术后的讨论交流。
上文所列出的详细说明仅仅是针对本发明的可行性实施方式的具体说明，它们并非用以限制本发明的保护范围，凡未脱离本发明技艺精神所作的等效实施方式或变更均应包含在本发明的保护范围之内。

Claims (13)

  1. 一种基于虚拟现实技术的医学影像处理***，其特征在于：包括相互通讯连接的控制单元、数据处理单元、显示单元、交互单元和传输单元，所述控制单元指令显示单元在VR设备中显示三维几何模型数据；所述交互单元通过实时获取交互命令，对VR设备中的三维几何模型进行变换；所述传输单元通过传输三维几何模型的变换参数，实现交互同步变换。
  2. 如权利要求1所述的基于虚拟现实技术的医学影像处理***，用于与VR设备配合显示，其特征在于：所述数据处理单元包括采集单元、分割单元和重建单元，所述采集单元用于获取医学影像数据；所述分割单元用于对医学图像数据中的目标区域进行分割，得到分割结果；所述重建单元用于对分割结果进行三维重建，生成三维几何模型数据。
  3. 如权利要求1所述的基于虚拟现实技术的医学影像处理***，其特征在于：所述显示单元包括用于显示医学图像的第一子单元、用于显示三维模型的第二子单元和用于VR设备显示的第三子单元。
  4. 一种基于虚拟现实技术的医学影像显示方法，其特征在于：包括如下步骤：
    S1.共享三维几何模型数据，使得VR设备自动加载并在其中渲染显示三维几何模型；
    S2.根据用户交互命令，对VR设备显示场景中的三维几何模型进行变换，得到变换参数；
    S3.VR设备根据变换参数，同步变换操作三维几何模型并进行显示。
  5. 如权利要求4所述的基于虚拟现实技术的医学影像显示方法，其特征在于：还包括如下步骤：S4.多个VR设备同时连接，下载同步重建网格数据，进入协同交互会话，并同步显示三维几何模型。
  6. 如权利要求5所述的基于虚拟现实技术的医学影像显示方法，其特征在于：还包括如下步骤：S5.对三维几何模型进行同步交互，通过当前VR设备对三维几何模型进行变换处理，处理后的参数同步发送至其他VR设备，其他VR设备将对应的三维几何模型变换处理后进行显示。
  7. 如权利要求6所述的基于虚拟现实技术的医学影像显示方法，其特征在于：在步骤S5中，包括步骤S51.在协同交互会话过程中，当前VR设备对三维几何模型进行拾取选中后，该三维几何模型被锁定，使得其他VR设备不能对该三维几何模型进行操作；当前VR设备对三维几何模型取消选中后，该三维几何模型解除锁定，并将解锁信息传送至其他VR设备。
  8. 如权利要求6所述的基于虚拟现实技术的医学影像显示方法，其特征在于：在步骤S5 中，包括步骤S53.在协同交互会话过程中，新增加的VR设备通过连接，下载同步重建网格数据，进入会话，并同步显示三维几何模型。
  9. 如权利要求6所述的基于虚拟现实技术的医学影像显示方法，其特征在于：在步骤S5中，包括步骤S54.在协同交互会话过程中，VR设备通过断开连接而随时退出会话，所有VR设备退出会话后，该会话关闭。
  10. 如权利要求4所述的基于虚拟现实技术的医学影像显示方法，其特征在于：还包括如下步骤，S0.获取医学影像及数据，并进行预处理；进行三维重建，得到三维几何模型数据。
  11. 一种基于虚拟现实技术的多人协同交互方法，其特征在于：包括以下步骤：
    S1.启动服务器，并连接多个VR设备；
    S2.启动各个VR设备向服务器验证信息；
    S3.下载同步重建网格数据，开始协同交互会话；
    S4.对三维几何模型进行同步交互，通过当前VR设备对三维几何模型进行变换处理，处理的参数通过服务器同步发给其他VR设备，其他VR设备会将对应的三维几何模型进行变换处理后进行显示。
  12. 如权利要求11所述的基于虚拟现实技术的多人协同交互方法，其特征在于：在步骤S4中，包括步骤S41.在协同交互会话过程中，当前VR设备对三维几何模型进行拾取选中后，该三维几何模型被锁定，使得其他VR设备不能对该三维几何模型进行操作；当前VR设备对三维几何模型取消选中后，该三维几何模型解除锁定，并将解锁信息传送至其他VR设备。
  13. 如权利要求11所述的基于虚拟现实技术的多人协同交互方法，其特征在于：在步骤S4中，包括步骤S42.在协同交互会话过程中，新增加的VR设备通过连接，下载同步重建网格数据，进入会话，并同步显示三维几何模型；VR设备通过断开连接而随时退出会话，所有VR设备退出会话后，该会话关闭。
PCT/CN2021/077839 2020-06-03 2021-02-25 基于虚拟现实技术的医学影像显示方法及*** WO2021244074A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010494407.X 2020-06-03
CN202010494407.XA CN111768491A (zh) 2020-06-03 2020-06-03 基于虚拟现实技术的医学影像显示方法及***

Publications (1)

Publication Number Publication Date
WO2021244074A1 true WO2021244074A1 (zh) 2021-12-09

Family

ID=72720075

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/077839 WO2021244074A1 (zh) 2020-06-03 2021-02-25 基于虚拟现实技术的医学影像显示方法及***

Country Status (2)

Country Link
CN (1) CN111768491A (zh)
WO (1) WO2021244074A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111768491A (zh) * 2020-06-03 2020-10-13 上海昕健医疗技术有限公司 基于虚拟现实技术的医学影像显示方法及***

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106806021A (zh) * 2016-11-21 2017-06-09 厦门强本宇康科技有限公司 一种基于人体器官3d模型的vr手术模拟***及方法
CN106898044A (zh) * 2017-02-28 2017-06-27 成都金盘电子科大多媒体技术有限公司 一种基于医学影像并利用vr技术的器官拆分和操作方法及***
US20180247449A1 (en) * 2017-02-28 2018-08-30 Medicalip Co., Ltd. Method and apparatus for controlling 3d medical image
CN109599025A (zh) * 2019-02-14 2019-04-09 西南石油大学 一种基于vr的多人协同钻井模拟***
CN110300547A (zh) * 2017-01-25 2019-10-01 全息眼株式会社 医疗信息虚拟现实服务器***，医疗信息虚拟现实程序，医疗信息虚拟现实***，医疗信息虚拟现实用数据的创建方法，以及医疗信息虚拟现实用数据
CN111768491A (zh) * 2020-06-03 2020-10-13 上海昕健医疗技术有限公司 基于虚拟现实技术的医学影像显示方法及***

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN202306552U (zh) * 2011-06-24 2012-07-04 海纳医信（北京）软件科技有限责任公司 远程实时会诊***
CN103106348A (zh) * 2013-03-08 2013-05-15 上海交通大学医学院附属第九人民医院 虚拟外科手术模拟方法及其装置
CN104112384A (zh) * 2013-04-22 2014-10-22 天津市天堰医教科技开发有限公司 一种神经外科虚拟手术训练***
US10154239B2 (en) * 2014-12-30 2018-12-11 Onpoint Medical, Inc. Image-guided surgery with surface reconstruction and augmented reality visualization
CN108280871A (zh) * 2017-01-03 2018-07-13 杭州英库医疗科技有限公司 医学影像三维重建***
CN108694726A (zh) * 2017-04-05 2018-10-23 贵州利体数字医学科技有限公司 医学影像三维重建***
CN107296650A (zh) * 2017-06-01 2017-10-27 西安电子科技大学 基于虚拟现实和增强现实的智能手术辅助***
CN109247985A (zh) * 2018-09-05 2019-01-22 北京数医科技有限公司 一种基于3d人体的手术辅助***及方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106806021A (zh) * 2016-11-21 2017-06-09 厦门强本宇康科技有限公司 一种基于人体器官3d模型的vr手术模拟***及方法
CN110300547A (zh) * 2017-01-25 2019-10-01 全息眼株式会社 医疗信息虚拟现实服务器***，医疗信息虚拟现实程序，医疗信息虚拟现实***，医疗信息虚拟现实用数据的创建方法，以及医疗信息虚拟现实用数据
CN106898044A (zh) * 2017-02-28 2017-06-27 成都金盘电子科大多媒体技术有限公司 一种基于医学影像并利用vr技术的器官拆分和操作方法及***
US20180247449A1 (en) * 2017-02-28 2018-08-30 Medicalip Co., Ltd. Method and apparatus for controlling 3d medical image
CN109599025A (zh) * 2019-02-14 2019-04-09 西南石油大学 一种基于vr的多人协同钻井模拟***
CN111768491A (zh) * 2020-06-03 2020-10-13 上海昕健医疗技术有限公司 基于虚拟现实技术的医学影像显示方法及***

Also Published As

Publication number Publication date
CN111768491A (zh) 2020-10-13

Similar Documents

Publication Publication Date Title
KR102018565B1 (ko) 수술 시뮬레이션 정보 구축 방법, 장치 및 프로그램
CN105992996B (zh) 外科手术环境中的动态和交互式导航
CN107296650A (zh) 基于虚拟现实和增强现实的智能手术辅助***
CN109036548A (zh) 基于混合现实三维可视化的辅助诊疗***
CN109464195A (zh) 双模式增强现实外科手术***和方法
CN106874700A (zh) 基于Web的手术模拟方法、手术模拟装置及电子设备
CN106569673A (zh) 多媒体病历报告的显示方法及多媒体病历报告的显示设备
DE19543410A1 (de) Virtuelles Untersuchungssystem für innere Hohlräume
EP0646263A1 (en) Computer graphic and live video system for enhancing visualisation of body structures during surgery
US20210353361A1 (en) Surgical planning, surgical navigation and imaging system
CN106890024A (zh) 一种肝脏肿瘤热消融手术的术前辅助装置
CN110021445A (zh) 一种基于vr模型的医疗***
CN113017832A (zh) 一种基于虚拟现实技术的穿刺手术模拟方法
Abou El-Seoud et al. An interactive mixed reality ray tracing rendering mobile application of medical data in minimally invasive surgeries
WO2021244074A1 (zh) 基于虚拟现实技术的医学影像显示方法及***
WO2021043684A1 (en) Method for analysing medical image data in a virtual multi-user collaboration, a computer program, a user interface and a system
CN111658142A (zh) 一种基于mr的病灶全息导航方法及***
Haigron et al. 3D navigation in medicine
US20230054394A1 (en) Device and system for multidimensional data visualization and interaction in an augmented reality virtual reality or mixed reality image guided surgery
CN113256820B (zh) 基于边缘检测的下颌面病变数字显影方法
CN111462314B (zh) 器官三维图像重建方法、手术导航方法及手术辅助***
CN112950774A (zh) 一种三维建模装置、手术规划***及教学***
Chen et al. A system design for virtual reality visualization of medical image
Stoyanov et al. Current issues of photorealistic rendering for virtual and augmented reality in minimally invasive surgery
CN201840505U (zh) 计算机辅助针刀定位装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21816807

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21816807

Country of ref document: EP

Kind code of ref document: A1