WO2022085701A1 - Three-dimensional medical image display method, server, system, and program - Google Patents
Three-dimensional medical image display method, server, system, and program Download PDFInfo
- Publication number
- WO2022085701A1 WO2022085701A1 PCT/JP2021/038679 JP2021038679W WO2022085701A1 WO 2022085701 A1 WO2022085701 A1 WO 2022085701A1 JP 2021038679 W JP2021038679 W JP 2021038679W WO 2022085701 A1 WO2022085701 A1 WO 2022085701A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- management information
- lump
- medical image
- record
- dimensional medical
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims description 27
- 230000004044 response Effects 0.000 claims abstract description 6
- 239000003550 marker Substances 0.000 claims description 17
- 239000000126 substance Substances 0.000 claims 1
- 230000002685 pulmonary effect Effects 0.000 abstract description 9
- 206010056342 Pulmonary mass Diseases 0.000 description 56
- 238000002591 computed tomography Methods 0.000 description 48
- 238000012545 processing Methods 0.000 description 34
- 238000010586 diagram Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 7
- 239000000284 extract Substances 0.000 description 5
- 238000001514 detection method Methods 0.000 description 4
- 230000007423 decrease Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 229940079593 drug Drugs 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
- A61B6/02—Arrangements for diagnosis sequentially in different planes; Stereoscopic radiation diagnosis
- A61B6/03—Computed tomography [CT]
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
Definitions
- the present invention relates to a three-dimensional medical image display method, a server, a system and a program.
- Non-Patent Document 1 a technique for automatically detecting a lung nodule from a CT (Computed Tomography) image using a neural network has been proposed. Further, as described in Non-Patent Document 2, for example, a technique for automatically tracking a lung nodule detected by a time-series CT image has also been proposed. With these techniques, it is possible to reduce the burden on the doctor who makes a diagnosis based on the CT image.
- CT Computer Tomography
- the present invention is a three-dimensional medical image display method, a server, which enables a doctor to present effective information based on the result of automatically tracking a mass detected automatically in chronological order.
- Systems and programs are intended to be provided.
- [1] Regarding the first mass according to the result of identifying the first mass automatically detected from the three-dimensional medical image and the second mass in which the record of management information is already stored.
- [2] The method for displaying a three-dimensional medical image according to [1], wherein the management information includes time-series information regarding the first and second lumps.
- [3] The method for displaying a three-dimensional medical image according to [2], wherein the display of management information includes a graph showing characteristic values of the first and second lumps in chronological order.
- [4] The method for displaying a three-dimensional medical image according to [3], wherein the display of management information includes a graph showing a plurality of sets of characteristic values of the first and second lumps at the same time in chronological order.
- [5] The three-dimensional medical use according to any one of [1] to [4], wherein the display of the management information is displayed together with the three-dimensional medical image and includes a marker indicating the position of the first or second mass. How to display the image.
- [6] The method for displaying a three-dimensional medical image according to [5], wherein the display of management information includes an ID display displayed in association with a marker.
- the display of management information is the three-dimensional medical image according to [5] or [6], which is displayed in association with a marker and includes a graph showing the characteristic values of the first and second lumps in chronological order. Display method.
- a terminal including a processor that implements a function of displaying information including a three-dimensional medical image on the display, a function of acquiring management information about a lump contained in the three-dimensional medical image, and a function of displaying the management information on the display.
- the device the database in which the records of management information are stored, and the first lumps automatically detected from the 3D medical image and the second lumps in which the records of management information are already stored in the database.
- a system that includes a server with a processor that implements the ability to automatically store a record of management information about a first chunk in a database, depending on the results identified.
- the first lump according to the result of identifying the first lump automatically detected from the three-dimensional medical image and the second lump whose management information record is already stored in the database.
- a program that enables a computer to automatically store a record of management information about an object in a database and to extract a record of management information in response to an inquiry from a terminal device.
- Lung nodules detected in the past are diagrams conceptually showing an example of adding records of different lung nodules. It is a figure which conceptually shows an example of adding a record of a new lung nodule. It is a figure which shows the 2nd example of the processing flow on the server side in the system shown in FIG.
- FIG. 1 is a diagram schematically showing a configuration of a system according to an embodiment of the present invention.
- the system 10 includes a terminal device 100 and a server 200.
- the terminal device 100 presents information including a CT image to the doctor.
- the CT image is an example of a three-dimensional medical image.
- the server 200 is connected to the terminal device 100 via a network.
- the server 200 may be arranged in the same facility as the terminal device 100, for example, or may be arranged outside and connected to the terminal device 100 via the Internet or the like. Both the terminal device 100 and the server 200 are implemented using a computer hardware configuration as described later.
- the terminal device 100 and the server 200 may be mounted by a single device, or may be mounted in a plurality of devices in a distributed manner. Hereinafter, each configuration will be further described.
- the terminal device 100 includes a viewer 110, an add-in 120, and a database 130.
- the viewer 110 is a function of displaying information including a CT image on a display, and is implemented by a processor that operates according to a program.
- the viewer 110 displays the CT image on the display according to the image data read from the database 130 and the display parameters such as GSPS (Grayscale Softcopy Presentation State).
- the viewer 110 causes the display to display nodule management information regarding lung nodules included in the CT image according to the data provided by the add-in 120.
- the lung nodule is an example of a mass contained in a three-dimensional medical image.
- the add-in 120 is a function of acquiring nodule management information from the server 200, and is implemented by a processor that operates according to a program like the viewer 110.
- the database 130 is implemented according to, for example, PACS (Picture Archiving and Communication System).
- the captured CT image data is transmitted to the server 200 in DICOM (Digital Imaging and Communications in Medicine) CT format, and the image data processed by the server 200 is received in DICOM PR format and stored in the database 130. Accumulated in time series.
- DICOM Digital Imaging and Communications in Medicine
- the server 200 includes an interface 210, a processing module 220, and a database 230.
- the CT image data is transmitted and received in the DICOM standard format, so that the interface 210 is implemented as a DICOM server interface by software.
- the processing module 220 is a function for executing processing related to CT images and nodule management information as described below, and is implemented by a processor that operates according to a program.
- the processing module 220 processes the CT image data received from the terminal device 100 via the interface 210, and transmits the processed data to the terminal device 100 via the interface 210.
- the database 230 automatically stores records of nodule management information regarding lung nodules in the form of a nodule management table as described later.
- the processing module 220 uses the data read from the database 230 in processing the CT image data, and stores the data obtained by processing the CT image data in the database 230.
- the processing module 220 extracts a record of the nodule management table stored in the database 230 in response to an inquiry from the add-in 120 of the terminal device 100, and transmits the record to the terminal device 100.
- FIG. 2 is a diagram conceptually showing an example of a nodule management table in the system shown in FIG.
- the nodule management table 231 is stored in the database 230 of the server 200.
- the nodule management table 231 includes items for patient ID, nodule ID, date, coordinates (X, Y, Z), length, CT value, area, and volume.
- the date, coordinates (X, Y, Z), length, CT value, area, and volume are examples of characteristic values of a mass.
- the patient ID and date can uniquely identify (a set of) CT images taken on a particular date for a particular patient.
- the nodule ID is a unique ID given to each of the lung nodules identified in the CT image.
- the coordinates (X, Y, Z) are information that identifies the position of the lung nodule in the CT image.
- the length is information indicating the size of the lung nodule measured in the CT image.
- the table may contain additional information, such as information identifying the site being imaged. Alternatively, some of the information exemplified above may be omitted. Further, an image ID that uniquely identifies (a set of) CT images may be given, and a table that associates the image ID with information that identifies a patient ID, a date, and a site may be separately provided.
- FIG. 3 is a diagram showing a first example of nodule management information displayed in the system shown in FIG.
- the nodule management information according to the first example includes a marker 111 displayed together with a CT image on the terminal device 100, an ID display 112, and a progress graph 113.
- the marker 111 and the ID display 112 are displayed based on the record (specified by the patient ID and date) corresponding to the CT image of the target in the nodule management table 231 (see FIG. 2).
- the marker 111 indicates the position of the lung nodule and is displayed based on the information of the coordinates (X, Y, Z) included in the record.
- the ID display 112 is displayed in association with the marker 111 based on the information of the nodule ID included in the record.
- the character string displayed as the ID display 112 and the nodule ID do not necessarily have to be the same. For example, even if a number newly assigned for display for each nodule ID is displayed as the ID display 112. good.
- the marker 111 and the ID display 112 may be displayed simultaneously for a plurality of lung nodules, or the marker 111 and the ID display 112 may be sequentially displayed for a range of lung nodules selected by using a pointing device or the like on the display. ..
- the progress graph 113 is displayed in the nodule management table 231 (see FIG. 2) based on the target CT image and a plurality of records (specified on condition of the patient ID) corresponding to the CT images captured in the past. Will be done.
- the progress graph 113 shows the length and CT value of the lung nodule displaying the marker 111 and the ID display 112 in chronological order.
- either the length or the CT value, or another piece of information contained in a record in the nodule management table 231 may be displayed in chronological order.
- the marker 111, the ID display 112 and the progress graph 113 are displayed for one lung nodule, but the marker 111, the ID display 112 and the progress graph 113 are simultaneously displayed for a plurality of sets of lung nodules. May be good.
- FIG. 4 is a diagram showing a second example of nodule management information displayed in the system shown in FIG.
- the nodule management information according to the second example includes a summary graph 114 displayed on the terminal device 100 separately from the CT image, for example, before or after the display of the CT image.
- the summary graph 114 is displayed based on the CT image of the subject and a plurality of records (specified subject to the patient ID) corresponding to the CT images captured in the past.
- the summary graph 114 shows the lengths of the plurality of lung nodules contained in the CT image in chronological order.
- the CT value or another information contained in the record of the nodule management table 231 may be displayed in chronological order.
- the summary graph 114 shows the rate of increase / decrease in lung nodule length 115.
- “+ 190.6%” indicates an increase of 190.6% when the past lung nodule length is 100%, and "-5.4%” decreases by 5.4% in the same case. Show that you are doing.
- “+ 190.6%” indicates an increase of 190.6% when the past lung nodule length is 100%, and "-5.4%” decreases by 5.4% in the same case. Show that you are doing.
- by presenting such a summary graph 114 before the doctor starts the interpretation of the CT image it is possible to predict the point of interest in the interpretation and efficiently proceed with the interpretation.
- FIG. 5 is a diagram showing an example of a processing flow on the terminal device side in the system shown in FIG.
- the add-in 120 of the terminal device 100 sends an inquiry to the processing module 220 of the server 200 on condition of the patient ID input or selected by the doctor (step S110), and the processing module 220 responds to the inquiry.
- the add-in 120 displays the summary graph 114 as shown in FIG. 4 on the viewer 110 using the received record (step S130).
- the add-in 120 uses the record received in step S120 to display the marker 111 and the ID display 112 as shown in FIG.
- step S150 When any of the lung nodule markers 111 is selected by a doctor's operation (step S160), the add-in 120 causes the viewer 110 to display the progress graph 113 of the lung nodule using the record received in step S120 (step S160). S170).
- FIG. 6 is a diagram showing a first example of a server-side processing flow in the system shown in FIG.
- the process of the first example is executed, for example, after the server 200 receives the newly captured CT image from the terminal device 100.
- the processing module 220 of the server 200 receives CT image data from the terminal device 100 (step S210), and automatically detects and measures lung nodules contained in the CT image (step S220).
- the automatic detection and measurement of lung nodules is described in, for example, Haichao Cao, et al., "A Two-Stage Convolutional Neural Networks for Lung Nodule Detection," IEEE Journal of Biomedical and Health Informatics, Jan 2020. You can use a global feature point extraction method that uses deep learning.
- the processing module 220 performs the following processing.
- the processing module 220 contains a lung nodule automatically detected from a new CT image (first lung nodule) and a lung nodule in which a record of nodule management information is already stored in the nodule management table 231 of the database 230. Automatically identify (second lung nodule). That is, the processing module 220 determines whether or not these lung nodules are the same (step S240).
- second lung nodule For identification of newly detected lung nodules and previously detected lung nodules, for example, Ayman El-Baz, et. Al., "Non-rigid registration techniques for automatic follow-up of lung nodules," "The method using non-rigid registration described in International Congress Series, Volume 1281, May 2005, Pages 1115-1120 can be used.
- a method using rigid body registration or sequential registration may be used.
- the processing module 220 automatically stores a record of nodule management information regarding the newly detected lung nodule in the database 230 according to the result of identifying the lung nodule in step S240. Specifically, if it is determined in step S240 that the newly detected lung nodule is the same as the previously detected lung nodule, the processing module 220 has been detected in the past, as shown in FIG. A record of a new nodule (first nodule) associated with the same nodule ID as the nodule (second nodule) is added to the nodule management table 231 (step S250).
- step S240 the first pulmonary nodule is different from the second pulmonary nodule (if there are a plurality of previously detected pulmonary nodules, none of them is the same as the newly detected pulmonary nodule). If determined, as shown in FIG. 8, the processing module 220 has a new nodule (first) associated with a new nodule ID that is different from the previously detected lung nodule (second nodule). The record of lung nodule) is added to the nodule management table 231 (step S260). If there is no CT image captured in the past, or no lung nodule is detected in the past CT image, the processing module 220 has a lung nodule associated with the new nodule ID, as shown in FIG. Is added to the nodule management table 231 (step S260).
- step S240 and step S250 or step S260 described above are executed for all the lung nodules automatically detected in step S220 (loop processing of step S230), the processing module 220 outputs the processed CT image data to the terminal device 100. (Step S270).
- FIG. 10 is a diagram showing a second example of the processing flow on the server side in the system shown in FIG.
- the process of the second example is executed when the server 200 receives an inquiry from the terminal device 100.
- the processing module 220 of the server 200 receives an inquiry on the condition of the patient ID from the terminal device 100 (step S310), and extracts a record including the target patient ID from the nodule management table 231 (step S320). .
- the processing module 220 extracts records according to those conditions.
- the processing module 220 may extract only records showing lung nodules of a predetermined length (eg, 5 mm or more) for a predetermined period (eg, within the last 5 years). Such a condition may be included in the inquiry from the terminal device 100, or the processing module 220 may automatically extract records under the above conditions according to the program or setting information.
- the processing module 220 transmits the extracted record to the terminal device 100 (step S330).
- effective information can be presented by a doctor based on the result of tracking the detected lung nodule in chronological order.
- textual qualitative representations are not always optimal because numerical information such as center coordinates, length and CT values are diagnostically essential.
- a technique is used in which numerical information on lung nodules automatically detected by analysis of CT images is stored in a nodule management table, and lung nodules detected in time-series CT images are automatically tracked.
- 10 ... system, 100 ... terminal device, 110 ... viewer, 111 ... marker, 112 ... ID display, 113 ... progress graph, 114 ... summary graph, 115 ... increase / decrease ratio, 120 ... add-in, 130 ... database, 200 ... server, 210 ... interface, 220 ... processing module, 230 ... database, 231 ... nodule management table.
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Medical Informatics (AREA)
- Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Radiology & Medical Imaging (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Optics & Photonics (AREA)
- Biomedical Technology (AREA)
- Primary Health Care (AREA)
- Biophysics (AREA)
- High Energy & Nuclear Physics (AREA)
- Epidemiology (AREA)
- Pathology (AREA)
- Physics & Mathematics (AREA)
- Heart & Thoracic Surgery (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- Veterinary Medicine (AREA)
- Medical Treatment And Welfare Office Work (AREA)
- Apparatus For Radiation Diagnosis (AREA)
Abstract
The present invention provides a terminal device equipped with a processor that has implemented therein: a function of displaying information inclusive of CT images on a display; a function of acquiring nodule management information about pulmonary nodules shown in CT images; and a function of displaying the nodule management information on the display. Also provided is a server that is provided with: a database which stores therein records of nodule management information regarding pulmonary nodules; and a processor which has a function of automatically storing in the database records of nodule management information regarding a first pulmonary nodule that has been automatically detected from CT images, according to results of matching between the first pulmonary nodule and a second pulmonary nodule having records of nodule management information thereof already stored in the database, and a function of extracting records of nodule management information in response to an inquiry made from the terminal device.
Description
本発明は、3次元医用画像の表示方法、サーバー、システムおよびプログラムに関する。
The present invention relates to a three-dimensional medical image display method, a server, a system and a program.
例えば非特許文献1に記載されているように、ニューラルネットワークを用いてCT(Computed Tomography)画像から肺結節を自動的に検出する技術が提案されている。また、例えば非特許文献2に記載されているように、時系列のCT画像で検出された肺結節を自動的に追跡する技術も提案されている。これらの技術によって、CT画像に基づいて診断を行う医師の負担を軽減することができる。
For example, as described in Non-Patent Document 1, a technique for automatically detecting a lung nodule from a CT (Computed Tomography) image using a neural network has been proposed. Further, as described in Non-Patent Document 2, for example, a technique for automatically tracking a lung nodule detected by a time-series CT image has also been proposed. With these techniques, it is possible to reduce the burden on the doctor who makes a diagnosis based on the CT image.
しかしながら、上記のような技術を用いて医師により効果的な情報を提示する方法については、これまで提案されていない。そこで、本発明は、自動的に検出された塊状物を自動的に時系列で追跡した結果に基づいて医師により効果的な情報を提示することを可能にする3次元医用画像の表示方法、サーバー、システムおよびプログラムを提供することを目的とする。
However, no method has been proposed so far for presenting more effective information to doctors using the above techniques. Therefore, the present invention is a three-dimensional medical image display method, a server, which enables a doctor to present effective information based on the result of automatically tracking a mass detected automatically in chronological order. , Systems and programs are intended to be provided.
[1]3次元医用画像から自動的に検出された第1の塊状物と管理情報のレコードが既に格納されている第2の塊状物とを同定した結果に応じて、第1の塊状物に関する管理情報のレコードをデータベースに自動的に格納するステップと、管理情報のレコードを抽出するステップと、ディスプレイに3次元医用画像とともに、または3次元医用画像の表示の前もしくは後に、管理情報を表示させるステップとを含む3次元医用画像の表示方法。
[2]管理情報は、第1および第2の塊状物に関する時系列の情報を含む、[1]に記載の3次元医用画像の表示方法。
[3]管理情報の表示は、第1および第2の塊状物の特性値を時系列で示すグラフを含む、[2]に記載の3次元医用画像の表示方法。
[4]管理情報の表示は、複数組の第1および第2の塊状物の特性値を同時に時系列で示すグラフを含む、[3]に記載の3次元医用画像の表示方法。
[5]管理情報の表示は、3次元医用画像とともに表示され第1または第2の塊状物の位置を示すマーカーを含む、[1]から[4]のいずれか1項に記載の3次元医用画像の表示方法。
[6]管理情報の表示は、マーカーに関連付けて表示されるID表示を含む、[5]に記載の3次元医用画像の表示方法。
[7]管理情報の表示は、マーカーに関連付けて表示され第1および第2の塊状物の特性値を時系列で示すグラフを含む、[5]または[6]に記載の3次元医用画像の表示方法。
[8]3次元医用画像に含まれる塊状物に関する管理情報のレコードが格納されるデータベースと、3次元医用画像から自動的に検出された第1の塊状物と管理情報のレコードが既にデータベースに格納されている第2の塊状物とを同定した結果に応じて、第1の塊状物に関する管理情報のレコードをデータベースに自動的に格納する機能、および端末装置からの問い合わせに応じて管理情報のレコードを抽出する機能を実装するプロセッサとを備えるサーバー。
[9]プロセッサは、第1の塊状物が第2の塊状物と同じであると判定された場合は第2の塊状物と同じIDに関連付けられた第1の塊状物の管理情報のレコードをデータベースに格納し、第1の塊状物が第2の塊状物とは異なると判定された場合は第2の塊状物とは異なるIDに関連付けられた第1の塊状物の管理情報のレコードをデータベースに自動的に格納する、[8]に記載のサーバー。
[10]ディスプレイに3次元医用画像を含む情報を表示させる機能、3次元医用画像に含まれる塊状物に関する管理情報を取得する機能、および管理情報をディスプレイに表示させる機能を実装するプロセッサを備える端末装置と、管理情報のレコードが格納されるデータベース、および3次元医用画像から自動的に検出された第1の塊状物と管理情報のレコードが既にデータベースに格納されている第2の塊状物とを同定した結果に応じて、第1の塊状物に関する管理情報のレコードをデータベースに自動的に格納する機能を実装するプロセッサを備えるサーバーとを含むシステム。
[11]3次元医用画像から自動的に検出された第1の塊状物と管理情報のレコードが既にデータベースに格納されている第2の塊状物とを同定した結果に応じて、第1の塊状物に関する管理情報のレコードをデータベースに自動的に格納する機能と、および端末装置からの問い合わせに応じて管理情報のレコードを抽出する機能とをコンピュータに実現させるためのプログラム。 [1] Regarding the first mass according to the result of identifying the first mass automatically detected from the three-dimensional medical image and the second mass in which the record of management information is already stored. The step of automatically storing the management information record in the database, the step of extracting the management information record, and displaying the management information on the display together with the 3D medical image or before or after the display of the 3D medical image. How to display a 3D medical image including steps.
[2] The method for displaying a three-dimensional medical image according to [1], wherein the management information includes time-series information regarding the first and second lumps.
[3] The method for displaying a three-dimensional medical image according to [2], wherein the display of management information includes a graph showing characteristic values of the first and second lumps in chronological order.
[4] The method for displaying a three-dimensional medical image according to [3], wherein the display of management information includes a graph showing a plurality of sets of characteristic values of the first and second lumps at the same time in chronological order.
[5] The three-dimensional medical use according to any one of [1] to [4], wherein the display of the management information is displayed together with the three-dimensional medical image and includes a marker indicating the position of the first or second mass. How to display the image.
[6] The method for displaying a three-dimensional medical image according to [5], wherein the display of management information includes an ID display displayed in association with a marker.
[7] The display of management information is the three-dimensional medical image according to [5] or [6], which is displayed in association with a marker and includes a graph showing the characteristic values of the first and second lumps in chronological order. Display method.
[8] A database in which a record of management information about a lump contained in a three-dimensional medical image is stored, and a record of a first lump and management information automatically detected from the three-dimensional medical image are already stored in the database. A function to automatically store a record of management information about the first lump in a database according to the result of identifying the second lump, and a record of management information in response to an inquiry from a terminal device. A server with a processor that implements the ability to extract.
[9] When the processor determines that the first lump is the same as the second lump, the processor records the management information of the first lump associated with the same ID as the second lump. When it is stored in the database and it is determined that the first lump is different from the second lump, the record of the management information of the first lump associated with the ID different from the second lump is stored in the database. The server according to [8], which is automatically stored in.
[10] A terminal including a processor that implements a function of displaying information including a three-dimensional medical image on the display, a function of acquiring management information about a lump contained in the three-dimensional medical image, and a function of displaying the management information on the display. The device, the database in which the records of management information are stored, and the first lumps automatically detected from the 3D medical image and the second lumps in which the records of management information are already stored in the database. A system that includes a server with a processor that implements the ability to automatically store a record of management information about a first chunk in a database, depending on the results identified.
[11] The first lump according to the result of identifying the first lump automatically detected from the three-dimensional medical image and the second lump whose management information record is already stored in the database. A program that enables a computer to automatically store a record of management information about an object in a database and to extract a record of management information in response to an inquiry from a terminal device.
[2]管理情報は、第1および第2の塊状物に関する時系列の情報を含む、[1]に記載の3次元医用画像の表示方法。
[3]管理情報の表示は、第1および第2の塊状物の特性値を時系列で示すグラフを含む、[2]に記載の3次元医用画像の表示方法。
[4]管理情報の表示は、複数組の第1および第2の塊状物の特性値を同時に時系列で示すグラフを含む、[3]に記載の3次元医用画像の表示方法。
[5]管理情報の表示は、3次元医用画像とともに表示され第1または第2の塊状物の位置を示すマーカーを含む、[1]から[4]のいずれか1項に記載の3次元医用画像の表示方法。
[6]管理情報の表示は、マーカーに関連付けて表示されるID表示を含む、[5]に記載の3次元医用画像の表示方法。
[7]管理情報の表示は、マーカーに関連付けて表示され第1および第2の塊状物の特性値を時系列で示すグラフを含む、[5]または[6]に記載の3次元医用画像の表示方法。
[8]3次元医用画像に含まれる塊状物に関する管理情報のレコードが格納されるデータベースと、3次元医用画像から自動的に検出された第1の塊状物と管理情報のレコードが既にデータベースに格納されている第2の塊状物とを同定した結果に応じて、第1の塊状物に関する管理情報のレコードをデータベースに自動的に格納する機能、および端末装置からの問い合わせに応じて管理情報のレコードを抽出する機能を実装するプロセッサとを備えるサーバー。
[9]プロセッサは、第1の塊状物が第2の塊状物と同じであると判定された場合は第2の塊状物と同じIDに関連付けられた第1の塊状物の管理情報のレコードをデータベースに格納し、第1の塊状物が第2の塊状物とは異なると判定された場合は第2の塊状物とは異なるIDに関連付けられた第1の塊状物の管理情報のレコードをデータベースに自動的に格納する、[8]に記載のサーバー。
[10]ディスプレイに3次元医用画像を含む情報を表示させる機能、3次元医用画像に含まれる塊状物に関する管理情報を取得する機能、および管理情報をディスプレイに表示させる機能を実装するプロセッサを備える端末装置と、管理情報のレコードが格納されるデータベース、および3次元医用画像から自動的に検出された第1の塊状物と管理情報のレコードが既にデータベースに格納されている第2の塊状物とを同定した結果に応じて、第1の塊状物に関する管理情報のレコードをデータベースに自動的に格納する機能を実装するプロセッサを備えるサーバーとを含むシステム。
[11]3次元医用画像から自動的に検出された第1の塊状物と管理情報のレコードが既にデータベースに格納されている第2の塊状物とを同定した結果に応じて、第1の塊状物に関する管理情報のレコードをデータベースに自動的に格納する機能と、および端末装置からの問い合わせに応じて管理情報のレコードを抽出する機能とをコンピュータに実現させるためのプログラム。 [1] Regarding the first mass according to the result of identifying the first mass automatically detected from the three-dimensional medical image and the second mass in which the record of management information is already stored. The step of automatically storing the management information record in the database, the step of extracting the management information record, and displaying the management information on the display together with the 3D medical image or before or after the display of the 3D medical image. How to display a 3D medical image including steps.
[2] The method for displaying a three-dimensional medical image according to [1], wherein the management information includes time-series information regarding the first and second lumps.
[3] The method for displaying a three-dimensional medical image according to [2], wherein the display of management information includes a graph showing characteristic values of the first and second lumps in chronological order.
[4] The method for displaying a three-dimensional medical image according to [3], wherein the display of management information includes a graph showing a plurality of sets of characteristic values of the first and second lumps at the same time in chronological order.
[5] The three-dimensional medical use according to any one of [1] to [4], wherein the display of the management information is displayed together with the three-dimensional medical image and includes a marker indicating the position of the first or second mass. How to display the image.
[6] The method for displaying a three-dimensional medical image according to [5], wherein the display of management information includes an ID display displayed in association with a marker.
[7] The display of management information is the three-dimensional medical image according to [5] or [6], which is displayed in association with a marker and includes a graph showing the characteristic values of the first and second lumps in chronological order. Display method.
[8] A database in which a record of management information about a lump contained in a three-dimensional medical image is stored, and a record of a first lump and management information automatically detected from the three-dimensional medical image are already stored in the database. A function to automatically store a record of management information about the first lump in a database according to the result of identifying the second lump, and a record of management information in response to an inquiry from a terminal device. A server with a processor that implements the ability to extract.
[9] When the processor determines that the first lump is the same as the second lump, the processor records the management information of the first lump associated with the same ID as the second lump. When it is stored in the database and it is determined that the first lump is different from the second lump, the record of the management information of the first lump associated with the ID different from the second lump is stored in the database. The server according to [8], which is automatically stored in.
[10] A terminal including a processor that implements a function of displaying information including a three-dimensional medical image on the display, a function of acquiring management information about a lump contained in the three-dimensional medical image, and a function of displaying the management information on the display. The device, the database in which the records of management information are stored, and the first lumps automatically detected from the 3D medical image and the second lumps in which the records of management information are already stored in the database. A system that includes a server with a processor that implements the ability to automatically store a record of management information about a first chunk in a database, depending on the results identified.
[11] The first lump according to the result of identifying the first lump automatically detected from the three-dimensional medical image and the second lump whose management information record is already stored in the database. A program that enables a computer to automatically store a record of management information about an object in a database and to extract a record of management information in response to an inquiry from a terminal device.
上記のような構成によれば、検出された塊状物を時系列で追跡した結果に基づいて医師により効果的な情報を提示することができる。
According to the above configuration, it is possible to present effective information to the doctor based on the result of tracking the detected lumps in chronological order.
以下に添付図面を参照しながら、本発明の好適な実施形態について詳細に説明する。なお、本明細書および図面において、実質的に同一の機能構成を有する構成要素については、同一の符号を付することにより重複した説明を省略する。
A preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings below. In the present specification and the drawings, components having substantially the same functional configuration are designated by the same reference numerals, and duplicate description will be omitted.
図1は、本発明の一実施形態に係るシステムの構成を概略的に示す図である。図示された例において、システム10は端末装置100とサーバー200とを含む。端末装置100は、CT画像を含む情報を医師に提示する。ここで、CT画像は3次元医用画像の例である。サーバー200は、ネットワークを介して端末装置100に接続される。サーバー200は、例えば端末装置100と同じ施設内に配置されもよいし、外部に配置されてインターネットなどを介して端末装置100に接続されてもよい。端末装置100およびサーバー200は、いずれも後述するようなコンピュータのハードウェア構成を用いて実装される。端末装置100およびサーバー200は、それぞれが単一の装置によって実装されてもよいし、複数の装置に分散して実装されてもよい。以下、それぞれの構成についてさらに説明する。
FIG. 1 is a diagram schematically showing a configuration of a system according to an embodiment of the present invention. In the illustrated example, the system 10 includes a terminal device 100 and a server 200. The terminal device 100 presents information including a CT image to the doctor. Here, the CT image is an example of a three-dimensional medical image. The server 200 is connected to the terminal device 100 via a network. The server 200 may be arranged in the same facility as the terminal device 100, for example, or may be arranged outside and connected to the terminal device 100 via the Internet or the like. Both the terminal device 100 and the server 200 are implemented using a computer hardware configuration as described later. The terminal device 100 and the server 200 may be mounted by a single device, or may be mounted in a plurality of devices in a distributed manner. Hereinafter, each configuration will be further described.
端末装置100は、ビューワー110と、アドイン120と、データベース130とを含む。ビューワー110は、ディスプレイにCT画像を含む情報を表示させる機能であり、プログラムに従って動作するプロセッサによって実装される。ビューワー110は、データベース130から読み込まれた画像データ、および例えばGSPS(Grayscale Softcopy Presentation State)などの表示パラメータに従ってCT画像をディスプレイに表示させる。また、ビューワー110は、アドイン120から提供されたデータに従って、CT画像に含まれる肺結節に関する結節管理情報をディスプレイに表示させる。ここで、肺結節は3次元医用画像に含まれる塊状物の例である。アドイン120は、サーバー200から結節管理情報を取得する機能であり、ビューワー110と同様にプログラムに従って動作するプロセッサによって実装される。データベース130は、例えばPACS(Picture Archiving and Communication System)に従って実装される。本実施形態では、撮像されたCT画像のデータがDICOM(Digital Imaging and Communications in Medicine) CT形式でサーバー200に送信され、サーバー200で処理された画像データがDICOM PR形式で受信され、データベース130に時系列で蓄積される。
The terminal device 100 includes a viewer 110, an add-in 120, and a database 130. The viewer 110 is a function of displaying information including a CT image on a display, and is implemented by a processor that operates according to a program. The viewer 110 displays the CT image on the display according to the image data read from the database 130 and the display parameters such as GSPS (Grayscale Softcopy Presentation State). Further, the viewer 110 causes the display to display nodule management information regarding lung nodules included in the CT image according to the data provided by the add-in 120. Here, the lung nodule is an example of a mass contained in a three-dimensional medical image. The add-in 120 is a function of acquiring nodule management information from the server 200, and is implemented by a processor that operates according to a program like the viewer 110. The database 130 is implemented according to, for example, PACS (Picture Archiving and Communication System). In this embodiment, the captured CT image data is transmitted to the server 200 in DICOM (Digital Imaging and Communications in Medicine) CT format, and the image data processed by the server 200 is received in DICOM PR format and stored in the database 130. Accumulated in time series.
サーバー200は、インターフェース210と、処理モジュール220と、データベース230とを含む。上記のように本実施形態ではCT画像のデータがDICOM規格の形式で送受信されるため、インターフェース210はDICOMサーバーインターフェースとしてソフトウェア的に実装される。処理モジュール220は、以下で説明するようなCT画像および結節管理情報に関する処理を実行する機能であり、プログラムに従って動作するプロセッサによって実装される。処理モジュール220は、端末装置100からインターフェース210を介して受信されたCT画像データを処理し、インターフェース210を介して処理後のデータを端末装置100に送信する。データベース230は、肺結節に関する結節管理情報のレコードを後述するような結節管理テーブルの形で自動的に格納する。処理モジュール220は、CT画像データを処理するにあたってデータベース230から読み込んだデータを利用し、またCT画像データを処理することによって得られたデータをデータベース230に格納する。処理モジュール220は、端末装置100のアドイン120からの問い合わせに応じてデータベース230に格納された結節管理テーブルのレコードを抽出して端末装置100に送信する。
The server 200 includes an interface 210, a processing module 220, and a database 230. As described above, in the present embodiment, the CT image data is transmitted and received in the DICOM standard format, so that the interface 210 is implemented as a DICOM server interface by software. The processing module 220 is a function for executing processing related to CT images and nodule management information as described below, and is implemented by a processor that operates according to a program. The processing module 220 processes the CT image data received from the terminal device 100 via the interface 210, and transmits the processed data to the terminal device 100 via the interface 210. The database 230 automatically stores records of nodule management information regarding lung nodules in the form of a nodule management table as described later. The processing module 220 uses the data read from the database 230 in processing the CT image data, and stores the data obtained by processing the CT image data in the database 230. The processing module 220 extracts a record of the nodule management table stored in the database 230 in response to an inquiry from the add-in 120 of the terminal device 100, and transmits the record to the terminal device 100.
図2は、図1に示すシステムにおける結節管理テーブルの例を概念的に示す図である。本実施形態において、結節管理テーブル231は、サーバー200のデータベース230に格納される。図示された例において、結節管理テーブル231は、患者ID、結節ID、日付、座標(X,Y,Z)、長さ、CT値、面積、および体積の項目を含む。ここで、日付、座標(X,Y,Z)、長さ、CT値、面積、および体積は、塊状物の特性値の例である。患者IDおよび日付によって、特定の患者について特定の日付に撮像されたCT画像(のセット)を一意に特定できる。結節IDは、CT画像において特定された肺結節のそれぞれに与えられる一意なIDである。座標(X,Y,Z)は、CT画像内の肺結節の位置を特定する情報である。長さは、CT画像において測定された肺結節の大きさを示す情報である。他の例では、撮像されている部位を識別する情報などの追加の情報がテーブルに含まれてもよい。あるいは、上記で例示した情報の一部が省略されてもよい。また、CT画像(のセット)を一意に特定する画像IDが与えられ、画像IDと患者ID、日付および部位を識別する情報を関連付けるテーブルが別途提供されてもよい。
FIG. 2 is a diagram conceptually showing an example of a nodule management table in the system shown in FIG. In this embodiment, the nodule management table 231 is stored in the database 230 of the server 200. In the illustrated example, the nodule management table 231 includes items for patient ID, nodule ID, date, coordinates (X, Y, Z), length, CT value, area, and volume. Here, the date, coordinates (X, Y, Z), length, CT value, area, and volume are examples of characteristic values of a mass. The patient ID and date can uniquely identify (a set of) CT images taken on a particular date for a particular patient. The nodule ID is a unique ID given to each of the lung nodules identified in the CT image. The coordinates (X, Y, Z) are information that identifies the position of the lung nodule in the CT image. The length is information indicating the size of the lung nodule measured in the CT image. In other examples, the table may contain additional information, such as information identifying the site being imaged. Alternatively, some of the information exemplified above may be omitted. Further, an image ID that uniquely identifies (a set of) CT images may be given, and a table that associates the image ID with information that identifies a patient ID, a date, and a site may be separately provided.
図3は、図1に示すシステムにおいて表示される結節管理情報の第1の例を示す図である。第1の例に係る結節管理情報は、端末装置100でCT画像とともに表示されるマーカー111と、ID表示112と、経過グラフ113とを含む。マーカー111およびID表示112は、結節管理テーブル231(図2参照)において対象のCT画像に対応するレコード(患者IDおよび日付を条件にして特定される)に基づいて表示される。具体的には、マーカー111は、肺結節の位置を示し、レコードに含まれる座標(X,Y,Z)の情報に基づいて表示される。ID表示112は、レコードに含まれる結節IDの情報に基づいて、マーカー111に関連付けて表示される。なお、ID表示112として表示される文字列と結節IDとは必ずしも同じでなくてもよく、例えばそれぞれの結節IDに対して表示用に新たに付番した番号がID表示112として表示されてもよい。複数の肺結節についてマーカー111およびID表示112が同時に表示されてもよいし、ディスプレイ上でポインティングデバイスなどを用いて選択された範囲の肺結節についてマーカー111およびID表示112が順次表示されてもよい。
FIG. 3 is a diagram showing a first example of nodule management information displayed in the system shown in FIG. The nodule management information according to the first example includes a marker 111 displayed together with a CT image on the terminal device 100, an ID display 112, and a progress graph 113. The marker 111 and the ID display 112 are displayed based on the record (specified by the patient ID and date) corresponding to the CT image of the target in the nodule management table 231 (see FIG. 2). Specifically, the marker 111 indicates the position of the lung nodule and is displayed based on the information of the coordinates (X, Y, Z) included in the record. The ID display 112 is displayed in association with the marker 111 based on the information of the nodule ID included in the record. The character string displayed as the ID display 112 and the nodule ID do not necessarily have to be the same. For example, even if a number newly assigned for display for each nodule ID is displayed as the ID display 112. good. The marker 111 and the ID display 112 may be displayed simultaneously for a plurality of lung nodules, or the marker 111 and the ID display 112 may be sequentially displayed for a range of lung nodules selected by using a pointing device or the like on the display. ..
一方、経過グラフ113は、結節管理テーブル231(図2参照)において対象のCT画像および過去に撮像されたCT画像に対応する複数のレコード(患者IDを条件にして特定される)に基づいて表示される。図示された例において、経過グラフ113は、マーカー111およびID表示112が表示されている肺結節の長さおよびCT値を時系列で示す。他の例では、長さおよびCT値のいずれか一方、または結節管理テーブル231のレコードに含まれる別の情報が時系列で表示されてもよい。また、図示された例では1つの肺結節についてマーカー111、ID表示112および経過グラフ113が表示されているが、複数組の肺結節についてマーカー111、ID表示112および経過グラフ113が同時に表示されてもよい。
On the other hand, the progress graph 113 is displayed in the nodule management table 231 (see FIG. 2) based on the target CT image and a plurality of records (specified on condition of the patient ID) corresponding to the CT images captured in the past. Will be done. In the illustrated example, the progress graph 113 shows the length and CT value of the lung nodule displaying the marker 111 and the ID display 112 in chronological order. In another example, either the length or the CT value, or another piece of information contained in a record in the nodule management table 231 may be displayed in chronological order. Further, in the illustrated example, the marker 111, the ID display 112 and the progress graph 113 are displayed for one lung nodule, but the marker 111, the ID display 112 and the progress graph 113 are simultaneously displayed for a plurality of sets of lung nodules. May be good.
図4は、図1に示すシステムにおいて表示される結節管理情報の第2の例を示す図である。第2の例に係る結節管理情報は、端末装置100でCT画像とは別に、例えばCT画像の表示の前または後に表示されるサマリーグラフ114を含む。サマリーグラフ114は、対象のCT画像および過去に撮像されたCT画像に対応する複数のレコード(患者IDを条件にして特定される)に基づいて表示される。図示された例において、サマリーグラフ114は、CT画像に含まれる複数の肺結節の長さを時系列で示す。他の例では、CT値、または結節管理テーブル231のレコードに含まれる別の情報が時系列で表示されてもよい。図示された例では、サマリーグラフ114に、肺結節の長さの増減比率115が表示されている。例えば、「+190.6%」は過去の肺結節長さを100%とした場合に190.6%拡大していることを示し、「-5.4%」は同じ場合に5.4%縮小していることを示す。例えば、このようなサマリーグラフ114を医師がCT画像の読影を開始する前に提示することによって、読影にあたって注目するポイントを予測し、効率的に読影を進めることが可能になる。
FIG. 4 is a diagram showing a second example of nodule management information displayed in the system shown in FIG. The nodule management information according to the second example includes a summary graph 114 displayed on the terminal device 100 separately from the CT image, for example, before or after the display of the CT image. The summary graph 114 is displayed based on the CT image of the subject and a plurality of records (specified subject to the patient ID) corresponding to the CT images captured in the past. In the illustrated example, the summary graph 114 shows the lengths of the plurality of lung nodules contained in the CT image in chronological order. In another example, the CT value or another information contained in the record of the nodule management table 231 may be displayed in chronological order. In the illustrated example, the summary graph 114 shows the rate of increase / decrease in lung nodule length 115. For example, "+ 190.6%" indicates an increase of 190.6% when the past lung nodule length is 100%, and "-5.4%" decreases by 5.4% in the same case. Show that you are doing. For example, by presenting such a summary graph 114 before the doctor starts the interpretation of the CT image, it is possible to predict the point of interest in the interpretation and efficiently proceed with the interpretation.
図5は、図1に示すシステムにおける端末装置側の処理フローの例を示す図である。図示された例において、端末装置100のアドイン120は、医師によって入力または選択された患者IDを条件にしてサーバー200の処理モジュール220に対する問い合わせを送信し(ステップS110)、問い合わせに応じて処理モジュール220がデータベース230に格納された結節管理テーブル231から抽出したレコードを受信する(ステップS120)。次に、アドイン120は、受信されたレコードを用いて、図4に示したようなサマリーグラフ114をビューワー110に表示させる(ステップS130)。医師の操作によってCT画像の読影が開始されると(ステップS140)、アドイン120はステップS120で受信されたレコードを用いて、図3に示したようなマーカー111およびID表示112をCT画像とともにビューワー110に表示させる(ステップS150)。医師の操作によっていずれかの肺結節のマーカー111が選択されると(ステップS160)、アドイン120はステップS120で受信されたレコードを用いてその肺結節の経過グラフ113をビューワー110に表示させる(ステップS170)。
FIG. 5 is a diagram showing an example of a processing flow on the terminal device side in the system shown in FIG. In the illustrated example, the add-in 120 of the terminal device 100 sends an inquiry to the processing module 220 of the server 200 on condition of the patient ID input or selected by the doctor (step S110), and the processing module 220 responds to the inquiry. Receives the record extracted from the node management table 231 stored in the database 230 (step S120). Next, the add-in 120 displays the summary graph 114 as shown in FIG. 4 on the viewer 110 using the received record (step S130). When the interpretation of the CT image is started by the operation of the doctor (step S140), the add-in 120 uses the record received in step S120 to display the marker 111 and the ID display 112 as shown in FIG. 3 together with the CT image. It is displayed on 110 (step S150). When any of the lung nodule markers 111 is selected by a doctor's operation (step S160), the add-in 120 causes the viewer 110 to display the progress graph 113 of the lung nodule using the record received in step S120 (step S160). S170).
図6は、図1に示すシステムにおけるサーバー側の処理フローの第1の例を示す図である。第1の例の処理は、例えばサーバー200が端末装置100から新たに撮像されたCT画像を受信した後に実行される。この例において、サーバー200の処理モジュール220は、端末装置100からCT画像データを受信し(ステップS210)、CT画像に含まれる肺結節の自動検出および自動計測を実施する(ステップS220)。ここで、肺結節の自動検出および自動計測には、例えばHaichao Cao, et al., "A Two-Stage Convolutional Neural Networks for Lung Nodule Detection," IEEE Journal of Biomedical and Health Informatics, Jan 2020に記載されているような、深層学習を使ったグローバルな特徴点抽出の手法を利用できる。ステップS220で自動検出された1または複数の肺結節のそれぞれについて、処理モジュール220は以下のような処理を実行する。
FIG. 6 is a diagram showing a first example of a server-side processing flow in the system shown in FIG. The process of the first example is executed, for example, after the server 200 receives the newly captured CT image from the terminal device 100. In this example, the processing module 220 of the server 200 receives CT image data from the terminal device 100 (step S210), and automatically detects and measures lung nodules contained in the CT image (step S220). Here, the automatic detection and measurement of lung nodules is described in, for example, Haichao Cao, et al., "A Two-Stage Convolutional Neural Networks for Lung Nodule Detection," IEEE Journal of Biomedical and Health Informatics, Jan 2020. You can use a global feature point extraction method that uses deep learning. For each of the one or more lung nodules automatically detected in step S220, the processing module 220 performs the following processing.
まず、処理モジュール220は、新たなCT画像から自動的に検出された肺結節(第1の肺結節)と、データベース230の結節管理テーブル231に結節管理情報のレコードが既に格納されている肺結節(第2の肺結節)とを自動的に同定する。つまり、処理モジュール220は、これらの肺結節が同じものであるか否かを判定する(ステップS240)。ここで、新たに検出された肺結節と過去に検出された肺結節との同定には、例えばAyman El-Baz, et. al., "Non-rigid registration techniques for automatic follow-up of lung nodules," International Congress Series, Volume 1281, May 2005, Pages 1115-1120に記載された非剛体レジストレーションを用いた手法を利用できる。その他にも、剛体レジストレーションや逐次レジストレーションを用いた手法を利用してもよい。
First, the processing module 220 contains a lung nodule automatically detected from a new CT image (first lung nodule) and a lung nodule in which a record of nodule management information is already stored in the nodule management table 231 of the database 230. Automatically identify (second lung nodule). That is, the processing module 220 determines whether or not these lung nodules are the same (step S240). Here, for identification of newly detected lung nodules and previously detected lung nodules, for example, Ayman El-Baz, et. Al., "Non-rigid registration techniques for automatic follow-up of lung nodules," "The method using non-rigid registration described in International Congress Series, Volume 1281, May 2005, Pages 1115-1120 can be used. In addition, a method using rigid body registration or sequential registration may be used.
処理モジュール220は、ステップS240で肺結節を同定した結果に応じて、新たに検出された肺結節に関する結節管理情報のレコードをデータベース230に自動的に格納する。具体的には、ステップS240において新たに検出された肺結節が過去に検出された肺結節と同じであると判定された場合、図7に示されるように、処理モジュール220は過去に検出された肺結節(第2の肺結節)と同じ結節IDに関連付けられた新たな肺結節(第1の肺結節)のレコードを結節管理テーブル231に追加する(ステップS250)。一方、ステップS240において第1の肺結節が第2の肺結節とは異なる(過去に検出された複数の肺結節がある場合、そのいずれもが新たに検出された肺結節と同じではない)と判定された場合、図8に示されるように、処理モジュール220は過去に検出された肺結節(第2の肺結節)とは異なる新たな結節IDに関連付けられた新たな肺結節(第1の肺結節)のレコードを結節管理テーブル231に追加する(ステップS260)。なお、過去に撮像されたCT画像がないか、または過去のCT画像では肺結節が検出されていない場合、図9に示されるように、処理モジュール220は新たな結節IDに関連付けられた肺結節のレコードを結節管理テーブル231に追加する(ステップS260)。
The processing module 220 automatically stores a record of nodule management information regarding the newly detected lung nodule in the database 230 according to the result of identifying the lung nodule in step S240. Specifically, if it is determined in step S240 that the newly detected lung nodule is the same as the previously detected lung nodule, the processing module 220 has been detected in the past, as shown in FIG. A record of a new nodule (first nodule) associated with the same nodule ID as the nodule (second nodule) is added to the nodule management table 231 (step S250). On the other hand, in step S240, the first pulmonary nodule is different from the second pulmonary nodule (if there are a plurality of previously detected pulmonary nodules, none of them is the same as the newly detected pulmonary nodule). If determined, as shown in FIG. 8, the processing module 220 has a new nodule (first) associated with a new nodule ID that is different from the previously detected lung nodule (second nodule). The record of lung nodule) is added to the nodule management table 231 (step S260). If there is no CT image captured in the past, or no lung nodule is detected in the past CT image, the processing module 220 has a lung nodule associated with the new nodule ID, as shown in FIG. Is added to the nodule management table 231 (step S260).
上記のステップS240およびステップS250またはステップS260の処理をステップS220で自動検出されたすべての肺結節について実行した(ステップS230のループ処理)後に、処理モジュール220は処理後のCT画像データを端末装置100に送信する(ステップS270)。
After the processing of step S240 and step S250 or step S260 described above is executed for all the lung nodules automatically detected in step S220 (loop processing of step S230), the processing module 220 outputs the processed CT image data to the terminal device 100. (Step S270).
図10は、図1に示すシステムにおけるサーバー側の処理フローの第2の例を示す図である。第2の例の処理は、サーバー200が端末装置100から問い合わせを受信したときに実行される。この例において、サーバー200の処理モジュール220は、端末装置100から患者IDを条件にした問い合わせを受信し(ステップS310)、対象の患者IDを含むレコードを結節管理テーブル231から抽出する(ステップS320)。ここで、端末装置100からの問い合わせに期間や長さなどの条件が追加で含まれる場合、処理モジュール220はそれらの条件に従ってレコードを抽出する。具体的には、処理モジュール220は所定の期間(例えば過去5年以内)の、所定の長さ(例えば5mm以上)の肺結節を示すレコードだけを抽出してもよい。このような条件は、端末装置100からの問い合わせに含まれていてもよいし、処理モジュール220側でプログラムまたは設定情報に従って自動的に上記のような条件でレコードを抽出してもよい。処理モジュール220は、抽出したレコードを端末装置100に送信する(ステップS330)。
FIG. 10 is a diagram showing a second example of the processing flow on the server side in the system shown in FIG. The process of the second example is executed when the server 200 receives an inquiry from the terminal device 100. In this example, the processing module 220 of the server 200 receives an inquiry on the condition of the patient ID from the terminal device 100 (step S310), and extracts a record including the target patient ID from the nodule management table 231 (step S320). .. Here, when the inquiry from the terminal device 100 additionally includes conditions such as a period and a length, the processing module 220 extracts records according to those conditions. Specifically, the processing module 220 may extract only records showing lung nodules of a predetermined length (eg, 5 mm or more) for a predetermined period (eg, within the last 5 years). Such a condition may be included in the inquiry from the terminal device 100, or the processing module 220 may automatically extract records under the above conditions according to the program or setting information. The processing module 220 transmits the extracted record to the terminal device 100 (step S330).
上記で説明したような本発明の一実施形態によれば、検出された肺結節を時系列で追跡した結果に基づいて、医師により効果的な情報を提示することができる。肺結節については、中心座標、長さおよびCT値などのような数値情報が診断上本質的であるため、テキストによる定性的な表現は必ずしも最適ではない。本実施形態では、CT画像の解析によって自動的に検出された肺結節に関する数値情報を結節管理テーブルに保存し、さらに時系列のCT画像で検出された肺結節を自動的に追跡する技術を用いて同じ結節IDのレコードを時系列で結節管理テーブルに追加することによって、例えば肺結節の位置を示すテキスト情報(上葉、中葉、背側、前側など)に基づく肺結節の同定にかかる労力や誤りの可能性がなくなる。また、例えば図3および図4に例示したような結節管理情報が表示されることによって、肺結節の長さなどの時系列変化の情報を容易に得ることができ、肺結節の検出結果に対する適切な判定をより迅速に行うことが可能になる。
According to one embodiment of the present invention as described above, effective information can be presented by a doctor based on the result of tracking the detected lung nodule in chronological order. For lung nodules, textual qualitative representations are not always optimal because numerical information such as center coordinates, length and CT values are diagnostically essential. In this embodiment, a technique is used in which numerical information on lung nodules automatically detected by analysis of CT images is stored in a nodule management table, and lung nodules detected in time-series CT images are automatically tracked. By adding records with the same nodule ID to the nodule management table in chronological order, for example, the effort required to identify lung nodules based on textual information indicating the position of lung nodules (upper lobe, middle lobe, dorsal side, anterior side, etc.) There is no possibility of mistakes. Further, for example, by displaying the nodule management information as illustrated in FIGS. 3 and 4, it is possible to easily obtain information on time-series changes such as the length of the lung nodule, which is appropriate for the detection result of the lung nodule. It becomes possible to make a clear judgment more quickly.
以上、添付図面を参照しながら本発明の好適な実施形態について詳細に説明したが、本発明はこれらの例に限定されない。本発明の属する技術の分野の当業者であれば、請求の範囲に記載された技術的思想の範疇内において、各種の変更例または修正例に想到し得ることは明らかであり、これらについても、当然に本発明の技術的範囲に属するものと了解される。
Although the preferred embodiments of the present invention have been described in detail with reference to the accompanying drawings, the present invention is not limited to these examples. It is clear that a person skilled in the art to which the present invention belongs can come up with various modifications or modifications within the scope of the technical ideas described in the claims. Naturally, it is understood that it belongs to the technical scope of the present invention.
10…システム、100…端末装置、110…ビューワー、111…マーカー、112…ID表示、113…経過グラフ、114…サマリーグラフ、115…増減比率、120…アドイン、130…データベース、200…サーバー、210…インターフェース、220…処理モジュール、230…データベース、231…結節管理テーブル。
10 ... system, 100 ... terminal device, 110 ... viewer, 111 ... marker, 112 ... ID display, 113 ... progress graph, 114 ... summary graph, 115 ... increase / decrease ratio, 120 ... add-in, 130 ... database, 200 ... server, 210 ... interface, 220 ... processing module, 230 ... database, 231 ... nodule management table.
Claims (11)
- 3次元医用画像から自動的に検出された第1の塊状物と管理情報のレコードが既に格納されている第2の塊状物とを同定した結果に応じて、前記第1の塊状物に関する前記管理情報のレコードをデータベースに自動的に格納するステップと、
前記管理情報のレコードを抽出するステップと、
ディスプレイに前記3次元医用画像とともに、または前記3次元医用画像の表示の前もしくは後に、前記管理情報を表示させるステップと
を含む3次元医用画像の表示方法。 The management regarding the first lump according to the result of identifying the first lump automatically detected from the three-dimensional medical image and the second lump in which the record of the management information is already stored. Steps to automatically store information records in the database,
The step of extracting the record of the management information and
A method of displaying a three-dimensional medical image including a step of displaying the management information on a display together with the three-dimensional medical image or before or after displaying the three-dimensional medical image. - 前記管理情報は、前記第1および第2の塊状物に関する時系列の情報を含む、請求項1に記載の3次元医用画像の表示方法。 The method for displaying a three-dimensional medical image according to claim 1, wherein the management information includes time-series information regarding the first and second lumps.
- 前記管理情報の表示は、前記第1および第2の塊状物の特性値を時系列で示すグラフを含む、請求項2に記載の3次元医用画像の表示方法。 The display of the management information is the method for displaying a three-dimensional medical image according to claim 2, which includes a graph showing the characteristic values of the first and second lumps in chronological order.
- 前記管理情報の表示は、複数組の前記第1および第2の塊状物の特性値を同時に時系列で示すグラフを含む、請求項3に記載の3次元医用画像の表示方法。 The method for displaying a three-dimensional medical image according to claim 3, wherein the display of the management information includes a graph showing a plurality of sets of characteristic values of the first and second lumps at the same time in chronological order.
- 前記管理情報の表示は、前記3次元医用画像とともに表示され前記第1または第2の塊状物の位置を示すマーカーを含む、請求項1から請求項4のいずれか1項に記載の3次元医用画像の表示方法。 The three-dimensional medical use according to any one of claims 1 to 4, wherein the display of the management information is displayed together with the three-dimensional medical image and includes a marker indicating the position of the first or second mass. How to display the image.
- 前記管理情報の表示は、前記マーカーに関連付けて表示されるID表示を含む、請求項5に記載の3次元医用画像の表示方法。 The method for displaying a three-dimensional medical image according to claim 5, wherein the display of the management information includes an ID display displayed in association with the marker.
- 前記管理情報の表示は、前記マーカーに関連付けて表示され前記第1および第2の塊状物の特性値を時系列で示すグラフを含む、請求項5または請求項6に記載の3次元医用画像の表示方法。 The three-dimensional medical image according to claim 5 or 6, wherein the display of the management information is displayed in association with the marker and includes a graph showing the characteristic values of the first and second lumps in chronological order. Display method.
- 3次元医用画像に含まれる塊状物に関する管理情報のレコードが格納されるデータベースと、
前記3次元医用画像から自動的に検出された第1の塊状物と、前記管理情報のレコードが既に前記データベースに格納されている第2の塊状物とを同定した結果に応じて、前記第1の塊状物に関する前記管理情報のレコードを前記データベースに自動的に格納する機能、および端末装置からの問い合わせに応じて前記管理情報のレコードを抽出する機能を実装するプロセッサと
を備えるサーバー。 A database that stores records of management information about lumps contained in 3D medical images, and
According to the result of identifying the first mass automatically detected from the three-dimensional medical image and the second mass in which the record of the management information is already stored in the database, the first mass is identified. A server including a processor that implements a function of automatically storing a record of the management information regarding a lump of the substance in the database and a function of extracting the record of the management information in response to an inquiry from a terminal device. - 前記プロセッサは、前記第1の塊状物が前記第2の塊状物と同じであると判定された場合は前記第2の塊状物と同じIDに関連付けられた前記第1の塊状物の管理情報のレコードを前記データベースに格納し、前記第1の塊状物が前記第2の塊状物とは異なると判定された場合は前記第2の塊状物とは異なるIDに関連付けられた前記第1の塊状物の管理情報のレコードを前記データベースに自動的に格納する、請求項8に記載のサーバー。 When the processor determines that the first lump is the same as the second lump, the processor manages the management information of the first lump associated with the same ID as the second lump. The record is stored in the database, and if it is determined that the first lump is different from the second lump, the first lump associated with an ID different from the second lump is associated with the first lump. The server according to claim 8, wherein the record of the management information of the above is automatically stored in the database.
- ディスプレイに3次元医用画像を含む情報を表示させる機能、前記3次元医用画像に含まれる塊状物に関する管理情報を取得する機能、および前記管理情報を前記ディスプレイに表示させる機能を実装するプロセッサを備える端末装置と、
前記管理情報のレコードが格納されるデータベース、および前記3次元医用画像から自動的に検出された第1の塊状物と前記管理情報のレコードが既に前記データベースに格納されている第2の塊状物とを同定した結果に応じて、前記第1の塊状物に関する前記管理情報のレコードを前記データベースに自動的に格納する機能を実装するプロセッサを備えるサーバーと
を含むシステム。 A terminal including a processor that implements a function of displaying information including a three-dimensional medical image on a display, a function of acquiring management information about a lump contained in the three-dimensional medical image, and a function of displaying the management information on the display. With the device
A database in which the record of the management information is stored, a first lump automatically detected from the three-dimensional medical image, and a second lump in which the record of the management information is already stored in the database. A system comprising a server comprising a processor that implements a function of automatically storing a record of said management information about the first chunk in the database, depending on the results of the identification. - 3次元医用画像から自動的に検出された第1の塊状物と管理情報のレコードが既にデータベースに格納されている第2の塊状物とを同定した結果に応じて、前記第1の塊状物に関する前記管理情報のレコードを前記データベースに自動的に格納する機能と、および端末装置からの問い合わせに応じて前記管理情報のレコードを抽出する機能と
をコンピュータに実現させるためのプログラム。 With respect to the first lump, depending on the result of identifying the first lump automatically detected from the three-dimensional medical image and the second lump whose management information record is already stored in the database. A program for realizing a function of automatically storing a record of the management information in the database and a function of extracting a record of the management information in response to an inquiry from a terminal device.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2020-176495 | 2020-10-21 | ||
JP2020176495A JP2022067737A (en) | 2020-10-21 | 2020-10-21 | Three-dimensional medical image display method, server, system and program |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022085701A1 true WO2022085701A1 (en) | 2022-04-28 |
Family
ID=81290574
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2021/038679 WO2022085701A1 (en) | 2020-10-21 | 2021-10-20 | Three-dimensional medical image display method, server, system, and program |
Country Status (2)
Country | Link |
---|---|
JP (1) | JP2022067737A (en) |
WO (1) | WO2022085701A1 (en) |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2012048395A (en) * | 2010-08-25 | 2012-03-08 | Fujifilm Corp | Medical information distribution device, medical information distribution method and medical information distribution program |
JP2013123528A (en) * | 2011-12-14 | 2013-06-24 | Hitachi Ltd | Image diagnosis support device and image diagnosis support method |
JP2017158913A (en) * | 2016-03-11 | 2017-09-14 | コニカミノルタ株式会社 | Image processing device and program |
JP2019037541A (en) * | 2017-08-25 | 2019-03-14 | 株式会社Aze | Medical image processing apparatus, control method for medical image processing apparatus, and program |
-
2020
- 2020-10-21 JP JP2020176495A patent/JP2022067737A/en active Pending
-
2021
- 2021-10-20 WO PCT/JP2021/038679 patent/WO2022085701A1/en active Application Filing
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2012048395A (en) * | 2010-08-25 | 2012-03-08 | Fujifilm Corp | Medical information distribution device, medical information distribution method and medical information distribution program |
JP2013123528A (en) * | 2011-12-14 | 2013-06-24 | Hitachi Ltd | Image diagnosis support device and image diagnosis support method |
JP2017158913A (en) * | 2016-03-11 | 2017-09-14 | コニカミノルタ株式会社 | Image processing device and program |
JP2019037541A (en) * | 2017-08-25 | 2019-03-14 | 株式会社Aze | Medical image processing apparatus, control method for medical image processing apparatus, and program |
Also Published As
Publication number | Publication date |
---|---|
JP2022067737A (en) | 2022-05-09 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20210158531A1 (en) | Patient Management Based On Anatomic Measurements | |
JP7225295B2 (en) | MEDICAL IMAGE DISPLAY APPARATUS, METHOD AND PROGRAM | |
US10568512B2 (en) | Control method of information terminal and computer-readable recording medium | |
JP6542004B2 (en) | Medical image processing apparatus and medical image processing system | |
US8588496B2 (en) | Medical image display apparatus, medical image display method and program | |
EP3100209B1 (en) | Extraction of information from an image and inclusion thereof in a clinical report | |
JP7082993B2 (en) | Medical image processing equipment, methods and programs, diagnostic support equipment, methods and programs, and medical support systems and methods | |
US8494242B2 (en) | Medical image management apparatus and method, and recording medium | |
US11468659B2 (en) | Learning support device, learning support method, learning support program, region-of-interest discrimination device, region-of-interest discrimination method, region-of-interest discrimination program, and learned model | |
US11139070B2 (en) | Medical information virtual reality server system, medical information virtual reality program, medical information virtual reality system, method of creating medical information virtual reality data, and medical information virtual reality data | |
JP2022520480A (en) | Image matching methods, devices, devices and storage media | |
US11688498B2 (en) | Medical document display control apparatus, medical document display control method, and medical document display control program | |
US20160093045A1 (en) | Medical image storage processing apparatus, method, and medium | |
JP2006149654A (en) | Support of diagnosis about lesion of eye fundus | |
JP2013027635A (en) | Image processing apparatus, and image processing method | |
JP7170000B2 (en) | LEARNING SYSTEMS, METHODS AND PROGRAMS | |
JP2015171456A (en) | Medical image display device, medical image display system, and medical image display program | |
WO2022085701A1 (en) | Three-dimensional medical image display method, server, system, and program | |
US11574726B2 (en) | Image analysis method for multicenter study and system thereof | |
CN111128321A (en) | Information display method and system, device, electronic equipment and readable medium | |
JP2017207793A (en) | Image display device and image display system | |
CN105451642A (en) | A system and method for real time analysis of medical imaging | |
US10885391B2 (en) | Image analysis apparatus, method, and program | |
CN112002406A (en) | Nodule display method based on confidence coefficient and computing equipment | |
JP7483018B2 (en) | Image processing device, image processing method and program, and image processing system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21882842 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 21882842 Country of ref document: EP Kind code of ref document: A1 |