CN112542234A

CN112542234A - System and method for organizing medical data

Info

Publication number: CN112542234A
Application number: CN202011458868.8A
Authority: CN
Inventors: 阿伦·因南耶; 阿比舍克·沙玛; 陈德仁
Original assignee: Shanghai United Imaging Intelligent Healthcare Co Ltd
Current assignee: Shanghai United Imaging Intelligent Healthcare Co Ltd
Priority date: 2019-12-17
Filing date: 2020-12-11
Publication date: 2021-03-23

Abstract

Methods and systems for organizing medical data are disclosed. For example, a computer-implemented method includes: receiving first data of a first data category, the first data having a first data format; extracting a plurality of first attributes from the first data using a first extractor; mapping the plurality of first attributes to a unified data format using a first mapper; receiving second data of a second data category, the second data having a second data format; extracting a plurality of second attributes from the second data using a second extractor; mapping the plurality of second attributes to a unified data format using a second mapper; and building an ontology for the use case by relating at least the plurality of first attributes and the plurality of second attributes.

Description

System and method for organizing medical data

Technical Field

Certain embodiments of the invention relate to medical data management. More specifically, some embodiments of the present invention provide methods and systems for managing medical data using relationship establishment. By way of example only, some embodiments of the invention are configured to manage radiological data and hospital data. It will be appreciated that the invention has broader applicability.

Background

Currently, many hospitals use PACS-based systems to store data, such as radiology data. PACS provides a relatively primitive query mechanism for users to search for data that limits the ability of radiologists and scientists to find the correct data, for example, for training purposes. In addition, medical patients often have a wide variety of data types associated with them, which are stored in different locations and in different formats, which makes it difficult for users to find the correct data of interest. Systems and methods with improved medical data organization are desirable, for example, for better searching and data recall.

Disclosure of Invention

In various embodiments, a computer-implemented method for organizing medical data, comprising: receiving first data of a first data category, the first data having a first data format; extracting a plurality of first attributes from the first data using a first extractor; mapping the plurality of first attributes to a unified data format using a first mapper; receiving second data of a second data category, the second data having a second data format; extracting a plurality of second attributes from the second data using a second extractor; mapping the plurality of second attributes to a unified data format using a second mapper; and building an ontology for the use case by relating at least the plurality of first attributes and the plurality of second attributes.

In various embodiments, a system for organizing medical data, comprising: a first data receiving module configured to receive first data of a first data category, the first data having a first data format; a first attribute extraction module configured to extract a plurality of first attributes from first data using a first extractor; a first attribute mapping module configured to map a plurality of first attributes to a unified data format using a first mapper; a second data receiving module configured to receive second data of a second data category, the second data having a second data format; a second attribute extraction module configured to extract a plurality of second attributes from second data using a second extractor; a second attribute mapping module configured to map a plurality of second attributes to a unified data format using a second mapper; and an ontology building module configured to build an ontology for the use case by relating at least the plurality of first attributes and the plurality of second attributes.

In various embodiments, a non-transitory computer-readable medium having stored thereon instructions that, when executed by a processor, cause the processor to perform one or more processes, comprising: receiving first data of a first data category, the first data having a first data format; extracting a plurality of first attributes from the first data using a first extractor; mapping the plurality of first attributes to a unified data format using a first mapper; receiving second data of a second data category, the second data having a second data format; extracting a plurality of second attributes from the second data using a second extractor; mapping the plurality of second attributes to a unified data format using a second mapper; and building an ontology for the use case by relating at least the plurality of first attributes and the plurality of second attributes.

According to embodiments, one or more benefits may be realized. These benefits and various additional objects, features and advantages of the present invention can be fully understood with reference to the detailed description and accompanying drawings that follow.

Drawings

Fig. 1 is a simplified diagram illustrating a system for managing medical data according to some embodiments.

Fig. 2 is a simplified diagram illustrating a method for managing medical data according to some embodiments.

FIG. 3 is a simplified diagram illustrating a computing system according to some embodiments.

Fig. 4 is a simplified diagram illustrating a neural network according to some embodiments.

FIG. 5 is a simplified diagram illustrating an ontology builder system according to some embodiments.

Fig. 6 is a simplified diagram illustrating the use of an ECG insert according to some embodiments.

FIG. 7 is a simplified diagram illustrating an ontology builder according to some embodiments.

Detailed Description

Fig. 1 is a simplified diagram illustrating a system for managing medical data according to some embodiments. This diagram is merely an example, which should not unduly limit the scope of the claims. One of ordinary skill in the art would recognize many variations, alternatives, and modifications. In some examples, system 10 includes a first data receiving module 12, a second data receiving module 14, a first attribute extraction module 16, a second attribute extraction module 18, a first attribute mapping module 20, a second attribute mapping module 22, and an ontology building module 24. In some examples, system 10 is configured to implement method S100 of fig. 2. Although a selected group of components has been shown above, many alternatives, modifications, and variations are possible. For example, some components may be expanded and/or combined. Some components may be eliminated. Other components may be inserted into the above components. The arrangement of components may be interchanged with other components substituted in accordance with the embodiment.

In various embodiments, the first data receiving module 12 is configured to receive first data of a first data category. In some examples, the first data has a first data format or form. In some examples, the first data category includes radiological data. In some examples, the first data includes one of a scan image and an annotation. In certain examples, the scan image includes a CT image, an MRI image, and/or a PET image. In some examples, the first data receiving module 12 is further configured to receive the first data through a first computer of a medical computer network.

In various embodiments, the second data receiving module 14 is configured to receive second data of a second data category. In some examples, the second data has a second data format or form. In some examples, the second data category includes profile data. In certain examples, the second data includes eating habits, lifestyle habits, physical data, and/or genetic history. In various examples, the second data category includes clinical data. In some examples, the second data includes electrocardiogram data, electroencephalography data, and/or blood report data. In some examples, the second data category includes diagnostic data. In certain examples, the second data includes a symptom, a disease, a condition, and/or a treatment plan. In some examples, the second data receiving module 14 is further configured to receive second data through a second computer of the computer network.

In various embodiments, the first attribute extraction module 16 is configured to extract a plurality of first attributes from the first data using a first extractor. In some examples, the first extractor is a machine learning model trained to extract attributes from data having the first data format. In some examples, first attribute extraction module 16 is configured to extract a plurality of first attributes at a first computer.

In various embodiments, second attribute extraction module 18 is configured to extract a plurality of second attributes from the second data using a second extractor. In some examples, the second extractor is a machine learning model trained to extract attributes from data having the second data format. In some examples, second attribute extraction module 18 is configured to extract a plurality of second attributes at the second computer.

In various embodiments, the first attribute mapping module 20 is configured to map the plurality of first attributes to a unified data format using a first mapper. In some examples, the unified data format is DICOM. In some examples, first attribute mapping module 20 is further configured to assign a uniform data label to each of the plurality of first attributes. In some examples, the first attribute mapping module is configured to map the first data at the first computer.

In various embodiments, the second attribute mapping module 22 is configured to map the plurality of second attributes to a unified data format using a second mapper. In some examples, second attribute mapping module 22 is further configured to assign a uniform data tag to each of the plurality of second attributes. In some examples, the first mapper and the first extractor are part of a first plug-in module. In some examples, the second mapper and the second extractor are part of a second plug-in module. In some examples, the second attribute mapping module 22 is configured to map the second data at the second computer.

In various embodiments, ontology building module 24 is configured to build an ontology for a use case by relating at least the plurality of first attributes and the plurality of second attributes.

In some embodiments, system 10 further includes a user input module configured to receive user input including one or more custom attribute relationships to be contacted. In some embodiments, system 10 also includes a graph builder (or alternatively, an ontology building module is used to generate an ontology graph based at least in part on the ontology to provide visual guidance to the user). In some embodiments, system 10 also includes a knowledge base configured to store the ontology in a searchable, sharable, and/or editable format. In some examples, the stored ontology is searchable by natural language. In some embodiments, the system 10 further includes a sharing module configured to share the mapped first data and the mapped second data with a computer network such that the first computer receives the mapped second data and the second computer receives the mapped first data.

Fig. 2 is a simplified diagram illustrating a method for managing medical data according to some embodiments. This diagram is merely an example, which should not unduly limit the scope of the claims. One of ordinary skill in the art would recognize many variations, alternatives, and modifications. In certain examples, the method S100 is implemented by the system 10 of fig. 1. In some examples, method S100 includes: a process S102 of receiving first data of a first data category, the first data having a first data format; a process S104 of extracting a plurality of first attributes from the first data using the first extractor; a process S106 of mapping the plurality of first attributes to a unified data format using a first mapper; a process S108 of receiving second data of a second data category, the second data having a second data format; a process S110 of extracting a plurality of second attributes from the second data using the second extractor; a process S112 of mapping the plurality of second attributes to the unified data format using a second mapper; and a process S114 of constructing an ontology for the use case by relating at least the plurality of first attributes and the plurality of second attributes. Although shown using a selected set of procedures for the method, many alternatives, modifications, and variations are possible. For example, some processes may be extended and/or combined. Other procedures may be inserted into the above-described procedure. Some processes may be eliminated. According to this embodiment, the order of the processes may be interchanged with other replaced processes.

In various embodiments, the process S102 of receiving first data of a first data category (the first data having a first data format) includes: the first data is received by a first computer of a medical computer network.

In various embodiments, the process S104 of extracting a plurality of first attributes from the first data using the first extractor includes: a plurality of first attributes is extracted at a first computer.

In various embodiments, the process S106 of mapping the plurality of first attributes to the unified data format using the first mapper includes: a uniform data tag is assigned to each of the plurality of first attributes. In some examples, mapping the plurality of first attributes includes: a plurality of first attributes is mapped at a first computer.

In various embodiments, the process S108 of receiving second data of a second data category (the second data having a second data format) includes: the second data is received by a second computer of the computer network.

In various embodiments, the process S110 of extracting a plurality of second attributes from the second data using the second extractor includes: a plurality of second attributes is extracted at a second computer.

In various embodiments, the process S112 of mapping the plurality of second attributes to the unified data format using the second mapper includes: a uniform data tag is assigned to each of the plurality of second attributes. In some examples, mapping the plurality of second attributes includes: the plurality of second attributes is mapped at the second computer.

In various embodiments, constructing the ontology for the use case by relating at least the plurality of first attributes and the plurality of second attributes S114 includes constructing the ontology for the cardiac patient.

In some examples, method S100 further includes: user input is received that includes one or more custom attribute relationships to contact. In some examples, receiving the user input includes: user input is received via the interactive graphical interface. In some examples, method S100 further includes: an ontology graph is generated based at least in part on the ontology to provide visual guidance to the user. In some examples, method S100 further includes: the ontology is stored in a searchable, sharable, and/or editable format. In some examples, the stored ontology is searchable in natural language. In some examples, method S100 further includes: the mapped first data and the mapped second data are shared with a computer network such that the first computer receives the mapped second data and the second computer receives the mapped first data.

In some examples, the unified data format is DICOM. In some examples, the first data category includes radiological data. In some examples, the first data includes one of a scan image and an annotation. In some examples, the scan image includes a CT image, an MRI image, and/or a PET image. In some examples, the second data category includes profile data. In certain examples, the second data includes eating habits, lifestyle habits, physical data, and/or genetic history. In some examples, the second data category includes clinical data. In some examples, the second data includes electrocardiogram data, electroencephalography data, and/or blood report data. In some examples, the second data category includes diagnostic data. In certain examples, the second data includes a symptom, a disease, a condition, and/or a treatment plan. In some examples, the first mapper and the first extractor are part of a first plug-in module. In some examples, the second mapper and the second extractor are part of a second plug-in module. In some examples, the first extractor is a machine learning model trained to extract attributes from data having the first data format. In some examples, the second extractor is a machine learning model trained to extract attributes from data having the second data format.

FIG. 3 is a simplified diagram illustrating a computing system according to some embodiments. This diagram is merely an example, which should not unduly limit the scope of the claims. One of ordinary skill in the art would recognize many variations, alternatives, and modifications. In some examples, computing system 6000 is a general purpose computing device. In some examples, computing system 6000 includes one or more processing units 6002 (e.g., one or more processors), one or more system memories 6004, one or more buses 6006, one or more input/output (I/O) interfaces 6008, and/or one or more network adapters 6012. In certain examples, one or more buses 6006 connect various system components, including, for example, one or more system memories 6004, one or more processing units 6002, one or more input/output (I/O) interfaces 6008, and/or one or more network adapters 6012. Although shown using a selected set of components for a computing system, many alternatives, modifications, and variations are possible. For example, some components may be expanded and/or combined. Other components may be inserted into the above components. Some components may be eliminated. The arrangement of components may be interchanged with other components substituted in accordance with the embodiment.

In some examples, computing system 6000 is a computer (e.g., server computer, client computer), smartphone, tablet, or wearable device. In some examples, some or all of the processes (e.g., steps) of method S100 are performed by computing system 6000. In some examples, some or all of the processes (e.g., steps) of method S100 are performed by one or more processing units 6002, which are directed by one or more codes. For example, the one or more codes are stored in one or more system memories 6004 (e.g., one or more non-transitory computer-readable media) and readable by computing system 6000 (e.g., readable by one or more processing units 6002). In various examples, one or more of system memory 6004 includes one or more computer-readable media in the form of volatile memory, such as Random Access Memory (RAM)6014, cache 6016, and/or storage system 6018 (e.g., floppy disks, CD-ROMs, and/or DVD-ROMs).

In some examples, one or more input/output (I/O) interfaces 6008 of computing system 6000 are configured to communicate with one or more external devices 6010 (e.g., keyboard, pointing device, and/or display). In some examples, one or more network adapters 6012 of computing system 6000 are configured to communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network (e.g., the internet)). In various examples, additional hardware and/or software modules are utilized in connection with computing system 6000, such as one or more microcode and/or one or more device drivers.

FIG. 4 is a simplified diagram illustrating a neural network according to some embodiments. For example, neural networks are used by one or more machine learning models. This diagram is merely an example, which should not unduly limit the scope of the claims. One of ordinary skill in the art would recognize many variations, alternatives, and modifications. The neural network 8000 is an artificial neural network. In some examples, the neural network 8000 includes an input layer 8002, one or more hidden layers 8004, and an output layer 8006. For example, the one or more hidden layers 8004 includes L neural network layers, including the 1 st neural network layer …, the ith neural network layer …, and the lth neural network layer, where L is a positive integer and i is an integer greater than or equal to 1 and less than or equal to L. Although shown using a selected set of components for a neural network, many alternatives, modifications, and variations are possible. For example, some components may be expanded and/or combined. Other components may be inserted into the above components. Some components may be eliminated. The arrangement of components may be interchanged with other components substituted in accordance with the embodiment.

In some examples, some or all of the processes (e.g., steps) of method S100 are performed by neural network 8000 (e.g., using computing system 6000). In certain examples, some or all of the processes (e.g., steps) of method S100 are performed by one or more processing units 6002, which are directed by one or more codes implementing neural network 8000. For example, one or more codes for the neural network 8000 are stored in one or more system memories 6004 (e.g., one or more non-transitory computer-readable media) and readable by the computing system 6000 (e.g., by the one or more processing units 6002).

In some examples, the neural network 8000 is a deep neural network (e.g., a convolutional neural network). In some examples, each neural network layer of the one or more hidden layers 8004 includes multiple sub-layers. As an example, the ith neural network layer includes a convolutional layer, an activation layer, and a pooling layer. For example, the convolutional layer is configured to perform feature extraction on an input (e.g., received by the input layer or from a previous neural network layer), the activation layer is configured to apply a non-linear activation function (e.g., a ReLU function) to an output of the convolutional layer, and the pooling layer is configured to compress (e.g., down-sample, e.g., by performing maximum pooling or average pooling) the output of the activation layer. As an example, output layer 8006 includes one or more fully connected layers.

As noted above and further emphasized here, fig. 4 is merely an example, and should not unduly limit the scope of the claims. One of ordinary skill in the art would recognize many variations, alternatives, and modifications. For example, the neural network 8000 is replaced with an algorithm that is not an artificial neural network. As an example, the neural network 8000 is replaced with a machine learning model that is not an artificial neural network.

FIG. 5 is a simplified diagram illustrating an ontology builder system according to some embodiments. In certain examples, the disclosed system is configured to receive Radiology Information System (RIS) data, such as Picture Archiving and Communication System (PACS) data and Hospital Information System (HIS) data. In some examples, PACS and HIS data in a hospital may be used to establish relationships in an ontology builder system. In certain examples, the disclosed systems are end-to-end systems configured to allow bridging of hospital information systems and radiology information systems (PACS), for example, to establish ontology-based systems around radiology information systems. In certain examples, the disclosed systems are configured to provide data for medical studies (e.g., novel medical studies) and/or training AI algorithms, e.g., via searching. In certain embodiments, the system of fig. 5 includes a HIS data receiving module, a RIS data receiving module, a knowledge extraction plug-in, a DICOM mapper, a relationship builder, a graph viewing module, a knowledge builder, an ontology repository, a search module, and an export module.

In certain embodiments, the HIS data receiving module is configured to receive HIS data, for example, from a HIS workflow, such as a database. In certain examples, the HIS data and/or HIS workflow correspond to patient enrollment, physician analysis, diagnosis, and/or treatment planning. In some examples, the HIS data is stored in one or more HIS formats (e.g., PDF).

In certain embodiments, the RIS data receiving module is configured to receive RIS data, for example, from an RIS workflow, such as a database. In certain examples, the RIS data and/or the RIS workflow correspond to scan images from a medical scanner (e.g., a CT, MR, or PET scanner). In certain examples, the RIS data is stored in one or more RIS formats.

In certain embodiments, the knowledge extraction plug-in is configured to extract data from HIS data, such as from various HIS data formats. In some examples, the knowledge extraction plug-in is further configured to transform the HIS data into a unified data format that is mappable to one or more DICOM attributes. In certain embodiments, the knowledge extraction plug-in is configured to extract data from the RIS data, such as from various RIS data formats. In some examples, the knowledge extraction plug-in is further configured to transform the RIS data into a unified data format that is mappable to one or more DICOM attributes.

In certain embodiments, a DICOM mapper (e.g., mapper module) is configured to map one or more attributes (e.g., extracted from HIS and/or HIS data) to one or more DICOM keys and/or convert data in a heterogeneous data format to a DICOM format, e.g., according to the DICOM standard.

In some embodiments, the relationship establisher is configured to establish one or more relationships between the plurality of DCIOM attributes. In some examples, the relationship builder is further configured to provide (e.g., to a user) a graphical user interface for inputting, contacting, and/or designing the relationship.

In some embodiments, the graphical viewing module is configured to present visual verification, such as editable visualizations (e.g., interactive graphs) for verification, for example, for complex attribute relationships.

In some embodiments, the knowledge builder is configured to receive the mapping relationships, for example, from a relationship builder. In some examples, the knowledge builder is configured to build a knowledge graph (e.g., an ontology graph), such as a final knowledge graph, based at least in part on a mapping relationship of one or more attributes extracted from the HIS data and the RIS data, for example.

In some embodiments, the ontology repository is configured to store one or more knowledge maps, such as one or more ontology maps. In some examples, the ontology repository is configured to store one or more ontology graphs in a searchable format (e.g., in a format that is recalled to modify established relationships).

In some embodiments, the search module is configured to provide a user interface for searching the established knowledge graph. In some examples, the results of the search include one or more sets of target DICOM objects (e.g., related DICOM objects) that can be viewed and/or passed to a training network in order to train a higher-level network (e.g., an artificial neural network).

In some embodiments, the output module is configured to output results of the search.

In certain embodiments, the disclosed system is configured to provide a user with an interactive ontology builder for HIS data and RIS data, for example, to fit HIS attributes and/or RIS attributes to DICOM, for example, to build a knowledge graph.

In some examples, the disclosed system is configured to provide a mechanism for applying (e.g., uploading) one or more plug-ins configured to extract and/or map unstructured data from the HIS and/or RIS format to the DICOM format (e.g., standard format).

In some examples, the disclosed systems are configured to provide (e.g., to a user) an ontology builder for DICOM data and/or provide a graphical viewer for designing relationships, e.g., by manipulating a relationship graph.

In certain embodiments, the disclosed system is configured to construct one or more knowledge graphs based on mapped attributes (e.g., extracted from HIS and/or RIS data) and/or to provide searchable data based on the constructed one or more knowledge graphs.

In certain embodiments, the disclosed system is configured to enable one or more users to pull a knowledge graph that includes attribute relationships, such as a knowledge graph that includes related DICOM objects in, for example, a cluster.

In certain examples, the disclosed system is applicable to medical prognostics. For example, a user who wants to know the prevailing conditions (e.g., all conditions) that may lead to glioblastoma may apply a plug-in through the system, which is configured to interpret one or more HIS records (e.g., extract data information) and/or pick up keywords and recommended scan(s) related to patient symptoms, and/or further combine findings with an ontology that has been built for the patient (e.g., including one or more annotations), for example, to report the identified relationships.

In certain embodiments, the disclosed system is configured to generate a knowledge graph, for example, based on existing data sets in a hospital database. In some examples, the system is configured for the user to run a search query in natural language, e.g., "how many% of patients with symptoms a, b, and c were diagnosed as having grade 3 glioblastoma and how does the tumor responded to treatment d? ". In some examples, the system is configured for a user to view search results, which may include one or more images, annotations, and/or to communicate data for training (e.g., training an artificial intelligence machine learning model) and/or advanced research.

In certain embodiments, the disclosed system is configured to receive and/or process PACS data and/or RIS data, such as data stored in DICOM, which may be associated with strict semantics, thereby making the data easily accessible. In some examples, the disclosed systems are configured to define one or more relationships between different types of data, e.g., based on defined (e.g., extracted) DICOM attributes.

In certain embodiments, the disclosed system is configured to receive an external query, for example from an external query system configured to utilize one or more relationships established based on HIS data and/or RIS data, for example to allow a user to establish simple queries, for example in natural language. For example, the query may be based on a deterministic relationship, such as "get all patients diagnosed with grade 3 glioblastoma". In some examples, the system is configured to output the one or more established relationships, e.g., in the form of an ontology graph, e.g., for training.

In certain embodiments, the disclosed system is configured to receive medical data stored on a PACS machine, including scans from CT, MRI, and the like. For example, the medical data may include annotations and reports in a structured format (e.g., DICOM) and prepared by a radiologist, such as results generated from performing diagnostics on RIS data. In some examples, the medical data may include treatment plans, such as suggestions for subsequent steps and/or additional scans of the treatment process. In certain examples, the disclosed systems are configured to support and assist a radiologist's diagnostic process, e.g., for anomaly detection and/or decision-making, e.g., by using artificial intelligence (e.g., deep learning).

In certain embodiments, the disclosed system is configured to provide data related to training one or more deep learning networks.

In certain embodiments, the disclosed system is configured to provide one or more knowledge graphs, for example, to help build ontologies based at least in part on well-structured data.

In certain embodiments, the disclosed systems are configured for ontology-based searching of PACS data, including, for example, the design and workflow to enhance user interaction with PACS-based systems and enable users to build complex queries around organized data, allowing them to gain deeper insight into the treated disease.

In certain examples, the disclosed system is configured for cardiac analysis. For example, the disclosed system is configured to assist in diagnosing a cardiac patient having patient data including eating habits, physical data (e.g., weight, BMI), daily activities, habits, and/or genetic history. In certain examples, the patient also has clinical data (e.g., ECG/EEG and/or blood reports) and/or radiological data (e.g., MR images and/or reports, and/or CT images and/or reports). In certain examples, the disclosed system is configured to help diagnose multiple patients with cardiac problems, such as patients with a wide variety of data types. For example, a patient may have data stored in various systems and in various formats. In certain examples, the disclosed systems are configured to consolidate (e.g., organize) various types of patient data (e.g., patient data saved in various formats and/or in various locations) and provide the data in a searchable manner in response to a user query (e.g., in natural language), such as "is an obese smoker led to increased heart strain? "," how likely a person with a heart strain of 0.43 on paragraph 3 is to be related to a particular diet or lifestyle? "and" early diagnosis: based on lifestyle and ECG/blood reports, can we ask if a patient is likely to have a particular disease? ". In certain examples, the disclosed system is configured to correlate various patient data including clinical diagnostic and radiological data.

Fig. 6 is a simplified diagram illustrating the use of an ECG insert according to some embodiments. In some examples, to build an ontology for use cases, the disclosed system is configured to tie one or more ontology attributes to the DICOM standard. In some examples, the disclosed system uses one or more plug-ins to extract information corresponding to one or more subsets of data (e.g., data from the HIS and RIS systems) into ontology properties. For example, the ECG plug-in fig. 6 illustrates an ECG plug-in configured to receive ECG data and/or ECG reports and output one or more DICOM attributes, which may correspond to one or more DICOM data tags.

FIG. 7 is a simplified diagram illustrating an ontology builder according to some embodiments. In some examples, the disclosed system is configured to combine data from various sources to establish ontological relationships and/or ontological diagrams. In certain examples, the disclosed system is configured to allow relationships to be established, for example between attributes extracted from various data (e.g., patient data, ECG data, diagnostic data, cardiac data). In some examples, the disclosed system is configured to map clinical data to DICOM attributes and to make the representation of the data consistent. In some examples, the disclosed system is configured to map DICOM attributes to ontologies. In some examples, the disclosed systems are configured to use one or more plug-ins to convert clinical and diagnostic data to DICOM, for example, to create ontological relationships and/or ontological diagrams.

In various embodiments, a computer-implemented method for organizing medical data, comprising: receiving first data of a first data category, the first data having a first data format; extracting a plurality of first attributes from the first data using a first extractor; mapping the plurality of first attributes to a unified data format using a first mapper; receiving second data of a second data category, the second data having a second data format; extracting a plurality of second attributes from the second data using a second extractor; mapping the plurality of second attributes to a unified data format using a second mapper; and building an ontology for the use case by relating at least the plurality of first attributes and the plurality of second attributes. In certain examples, the computer-implemented methods are performed by one or more processors. In some examples, the computer-implemented method is implemented at least in part according to method S100 of fig. 2. In certain examples, the method is implemented at least in part by the system 10 of fig. 1.

In some embodiments, the unified data format is DICOM.

In some embodiments, mapping the plurality of first attributes to the unified data format using the first mapper comprises: a uniform data tag is assigned to each of the plurality of first attributes. In some examples, mapping the plurality of second attributes to the unified data format using the second mapper includes: a uniform data tag is assigned to each of the plurality of second attributes.

In some embodiments, the first data class includes radiological data. In some examples, the first data includes one of a scan image and an annotation.

In some embodiments, the scan image comprises a CT image, an MRI image, and/or a PET image.

In some embodiments, the second data category includes profile data. In certain examples, the second data includes eating habits, lifestyle habits, physical data, and/or genetic history.

In some embodiments, the second data category includes clinical data. In some examples, the second data includes electrocardiogram data, electroencephalography data, and/or blood report data.

In some embodiments, the second data category includes diagnostic data. In certain examples, the second data includes a symptom, a disease, a condition, and/or a treatment plan.

In some embodiments, the first mapper and the first extractor are part of a first plug-in module. In some examples, the second mapper and the second extractor are part of a second plug-in module.

In some embodiments, the first extractor is a machine learning model trained to extract attributes from data having the first data format. In some examples, the second extractor is a machine learning model trained to extract attributes from data having the second data format.

In some embodiments, the method further comprises: user input is received that includes one or more custom attribute relationships to contact.

In some embodiments, receiving the user input comprises: user input is received via the interactive graphical interface.

In some embodiments, the method further comprises: an ontology graph is generated based at least in part on the ontology to provide visual guidance to the user.

In some embodiments, the method further comprises: the ontology is stored in a searchable, sharable, and/or editable format.

In some embodiments, the stored ontology is searchable by natural language.

In some embodiments, receiving the first data of the first data class comprises: the first data is received by a first computer of a medical computer network. In some examples, receiving the second data of the second data category includes: the second data is received by a second computer of the computer network.

In some embodiments, extracting the plurality of first attributes comprises: extracting, at a first computer, a plurality of first attributes; mapping the plurality of first attributes includes: mapping, at a first computer, a plurality of first attributes; extracting the plurality of second attributes comprises: extracting, at the second computer, a plurality of second attributes; and mapping the plurality of second attributes comprises: the plurality of second attributes is mapped at the second computer.

In some embodiments, the method further comprises: the mapped first data and the mapped second data are shared with a computer network such that the first computer receives the mapped second data and the second computer receives the mapped first data.

In various embodiments, a system for organizing medical data, comprising: a first data receiving module configured to receive first data of a first data category, the first data having a first data format; a first attribute extraction module configured to extract a plurality of first attributes from first data using a first extractor; a first attribute mapping module configured to map a plurality of first attributes to a unified data format using a first mapper; a second data receiving module configured to receive second data of a second data category, the second data having a second data format; a second attribute extraction module configured to extract a plurality of second attributes from second data using a second extractor; a second attribute mapping module configured to map a plurality of second attributes to a unified data format using a second mapper; and an ontology building module configured to build an ontology for the use case by relating at least the plurality of first attributes and the plurality of second attributes. In some examples, the system is implemented at least in part in accordance with the system 10 of fig. 1. In some examples, the system is configured to perform, at least in part, the method S100 of fig. 2.

In some embodiments, the unified data format is DICOM.

In some embodiments, the first attribute mapping module is further configured to assign a uniform data tag to each of the plurality of first attributes. In some examples, the second attribute mapping module is further configured to assign a uniform data tag to each of the plurality of second attributes.

In some embodiments, the system further comprises a user input module configured to receive user input comprising one or more custom attribute relationships to be contacted.

In some embodiments, the system further comprises a graph builder (or alternatively, an ontology building module is used to generate an ontology graph based at least in part on the ontology to provide visual guidance to the user).

In some embodiments, the system further comprises a knowledge base configured to store the ontology in a searchable, sharable, and/or editable format.

In some embodiments, the stored ontology is searchable by natural language.

In some embodiments, the first data receiving module is further configured to receive the first data through a first computer of a medical computer network. In some examples, the second data receiving module is further configured to receive second data through a second computer of the computer network.

In some embodiments, the first attribute extraction module is configured to extract a plurality of first attributes at the first computer. In some examples, the first attribute mapping module is configured to map the first data at the first computer. In some examples, the second attribute extraction module is configured to extract a plurality of second attributes at the second computer. In some examples, the second attribute mapping module is configured to map the second data at the second computer.

In some embodiments, the system further includes a sharing module configured to share the mapped first data and the mapped second data with a computer network such that the first computer receives the mapped second data and the second computer receives the mapped first data.

In various embodiments, a non-transitory computer-readable medium having stored thereon instructions that, when executed by a processor, cause the processor to perform one or more processes, comprising: receiving first data of a first data category, the first data having a first data format; extracting a plurality of first attributes from the first data using a first extractor; mapping the plurality of first attributes to a unified data format using a first mapper; receiving second data of a second data category, the second data having a second data format; extracting a plurality of second attributes from the second data using a second extractor; mapping the plurality of second attributes to a unified data format using a second mapper; and building an ontology for the use case by relating at least the plurality of first attributes and the plurality of second attributes. In some examples, the non-transitory computer-readable medium having instructions stored thereon is implemented according to method S100 of fig. 2. In some examples, a non-transitory computer-readable medium having instructions stored thereon is configured to be at least partially implemented by the system 10 (e.g., terminal) of fig. 1.

In some embodiments, the unified data format is DICOM.

In some embodiments, a non-transitory computer-readable medium having instructions stored thereon, which when executed by a processor, cause the processor to perform one or more processes comprising: a uniform data tag is assigned to each of the plurality of first attributes and/or a uniform data tag is assigned to each of the plurality of second attributes.

In some embodiments, a non-transitory computer-readable medium having instructions stored thereon, which when executed by a processor, cause the processor to perform one or more processes comprising: user input is received that includes one or more custom attribute relationships to contact.

In some embodiments, a non-transitory computer-readable medium having instructions stored thereon, which when executed by a processor, cause the processor to perform one or more processes comprising: user input is received via the interactive graphical interface.

In some embodiments, a non-transitory computer-readable medium having instructions stored thereon, which when executed by a processor, cause the processor to perform one or more processes comprising: an ontology graph is generated based at least in part on the ontology to provide visual guidance to the user.

In some embodiments, a non-transitory computer-readable medium having instructions stored thereon, which when executed by a processor, cause the processor to perform one or more processes comprising: the ontology is stored in a searchable, sharable, and/or editable format.

In some embodiments, the stored ontology is searchable by natural language.

In some embodiments, a non-transitory computer-readable medium having instructions stored thereon, which when executed by a processor, cause the processor to perform one or more processes comprising: the first data is received by a first computer of the medical computer network and/or the second data is received by a second computer of the computer network.

In some embodiments, a non-transitory computer-readable medium having instructions stored thereon, which when executed by a processor, cause the processor to perform one or more processes comprising: extracting, at a first computer, a plurality of first attributes; mapping, at a first computer, a plurality of first attributes; extracting, at the second computer, a plurality of second attributes; and mapping the plurality of second attributes at the second computer.

In some embodiments, a non-transitory computer-readable medium having instructions stored thereon, which when executed by a processor, cause the processor to perform one or more processes comprising: the mapped first data and the mapped second data are shared with a computer network such that the first computer receives the mapped second data and the second computer receives the mapped first data.

For example, some or all of the components of the various embodiments of the invention may be implemented using one or more software components, one or more hardware components, and/or one or more combinations of software and hardware components, each alone and/or in combination with at least one other component. In another example, some or all of the components of the various embodiments of the present invention are implemented in one or more circuits (e.g., one or more analog circuits and/or one or more digital circuits), each alone and/or in combination with at least one other component. In yet another example, although the above-described embodiments refer to particular features, the scope of the present invention also includes embodiments having different combinations of features and embodiments that do not include all of the described features. In yet another example, various embodiments and/or examples of the invention may be combined.

Further, the methods and systems described herein may be implemented on many different types of processing devices by program code comprising program instructions executable by a device processing subsystem. The software program instructions may include source code, object code, machine code, or any other data stored that is operable to cause a processing system to perform the methods and operations described herein. However, other embodiments may be used, such as firmware or even suitably designed hardware configured to perform the methods and systems described herein.

Data (e.g., associations, mappings, data inputs, data outputs, intermediate data results, final data results, etc.) for these systems and these methods may be stored and implemented in one or more different types of computer-implemented data stores, such as different types of storage devices and programming structures (e.g., RAM, ROM, EEPROM, flash memory, flat files, databases, programmed data structures, programmed variables, IF-THEN (or similar types) statement structures, application programming interfaces, etc.). It should be noted that data structures describe formats for organizing and storing data in databases, programs, memory, or other computer-readable media for use by a computer program.

The systems and methods may be provided on many different types of computer-readable media including computer storage mechanisms (e.g., CD-ROM, floppy disk, RAM, flash memory, computer hard drive, DVD, etc.) that contain instructions (e.g., software) for execution by a processor to perform the operations of the methods described herein and to implement the systems. The computer components, software modules, functions, data stores, and data structures described herein may be interconnected, directly or indirectly, to allow the flow of data required for their operation. It is further noted that a module or processor comprises code units performing software operations and may for example be implemented as subroutine units of code, or as software functional units of code, or as objects, such as object-oriented paradigms, or as applets, or as computer script language, or as other types of computer code. The software components and/or functionality may be located on one computer or distributed across multiple computers, depending on the particular situation.

The computing system may include a client device and a server. A client device and server are generally remotely located from each other and typically interact through a communication network. The relationship of client device and server arises by virtue of computer programs running on the respective computers and having an interrelationship between client device and server.

This description contains many specific embodiment details. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations, one or more features from a combination can in some cases be excised from the combination, and the combination may be directed to a subcombination or variation of a subcombination.

Also, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous. Moreover, in the embodiments described above, the separation of various system components should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated within a single software product or packaged into multiple software products.

While specific embodiments of the invention have been described, those skilled in the art will appreciate that there are other embodiments that are equivalent to the described embodiments. Therefore, it should be understood that the invention should not be limited to the particular embodiments shown.

Claims

1. A computer-implemented method for organizing medical data, the method comprising:

receiving first data of a first data category, the first data having a first data format;

extracting a plurality of first attributes from the first data using a first extractor;

mapping the plurality of first attributes to a unified data format using a first mapper;

receiving second data of a second data category, the second data having a second data format;

extracting a plurality of second attributes from the second data using a second extractor;

mapping the plurality of second attributes to the unified data format using a second mapper; and

building an ontology for the use case by relating at least the plurality of first attributes and the plurality of second attributes.

2. The computer-implemented method of claim 1,

the mapping the plurality of first attributes to a unified data format using a first mapper comprises: assigning a uniform data tag to each attribute of the plurality of first attributes; and is

The mapping the plurality of second attributes to a unified data format using a second mapper comprises: assigning a uniform data tag to each attribute of the plurality of second attributes.

3. The computer-implemented method of claim 1,

the first data category includes radiological data; and is

The first data includes one of a scan image and an annotation.

4. The computer-implemented method of claim 1,

the second data category includes profile data; and is

The second data comprises one of dietary habits, lifestyle habits, physical data, and genetic history; or

The second data category includes clinical data; and is

The second data comprises one of electrocardiogram data, electroencephalogram data, and blood report data; or

The second data category comprises diagnostic data; and is

The second data includes one of a symptom, a disease, a condition, and a treatment plan.

5. The computer-implemented method of claim 1,

the first mapper and the first extractor are part of a first plug-in module, and

the second mapper and the second extractor are part of a second plug-in module; or

The first extractor is a machine learning model trained to extract attributes from data having the first data format, and,

the second extractor is a machine learning model trained to extract attributes from data having the second data format.

6. The computer-implemented method of claim 1, further comprising:

receiving user input comprising one or more custom attribute relationships to contact, wherein the receiving user input comprises: user input is received via the interactive graphical interface.

7. The computer-implemented method of claim 1, further comprising: generating an ontology graph based at least in part on the ontology to provide visual guidance to a user; further comprising: storing the ontology in a searchable, sharable, and editable format, wherein the stored ontology is searchable in natural language.

8. The computer-implemented method of claim 1,

the receiving first data of a first data class comprises: receiving the first data by a first computer of a medical computer network; and is

Said receiving second data of a second data category comprises: receiving the second data over a second computer of the computer network, wherein,

the extracting the plurality of first attributes comprises: extracting, at the first computer, the plurality of first attributes;

the mapping the plurality of first attributes comprises: mapping the plurality of first attributes at the first computer;

the extracting the plurality of second attributes comprises: extracting, at the second computer, the plurality of second attributes; and is

The mapping the plurality of second attributes comprises: mapping the plurality of second attributes at the second computer,

wherein, the method further comprises:

sharing the mapped first data and the mapped second data with the computer network such that the first computer receives the mapped second data and the second computer receives the mapped first data.

9. A system for organizing medical data, the system comprising:

a first data receiving module configured to receive first data of a first data category, the first data having a first data format;

a first attribute extraction module configured to extract a plurality of first attributes from the first data using a first extractor;

a first attribute mapping module configured to map the plurality of first attributes to a unified data format using a first mapper;

a second data receiving module configured to receive second data of a second data category, the second data having a second data format;

a second attribute extraction module configured to extract a plurality of second attributes from the second data using a second extractor;

a second attribute mapping module configured to map the plurality of second attributes to the unified data format using a second mapper; and

an ontology building module configured to build an ontology for the use case by relating at least the plurality of first attributes and the plurality of second attributes.

10. A non-transitory computer-readable medium for organizing medical data, having instructions stored thereon, which when executed by a processor, cause the processor to perform a process, comprising: