CN112988382B - Medical image intelligent analysis system based on distributed deep learning - Google Patents
Medical image intelligent analysis system based on distributed deep learning Download PDFInfo
- Publication number
- CN112988382B CN112988382B CN202110268700.9A CN202110268700A CN112988382B CN 112988382 B CN112988382 B CN 112988382B CN 202110268700 A CN202110268700 A CN 202110268700A CN 112988382 B CN112988382 B CN 112988382B
- Authority
- CN
- China
- Prior art keywords
- model
- server
- neural network
- training
- distributed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 43
- 238000013135 deep learning Methods 0.000 title claims abstract description 40
- 238000012549 training Methods 0.000 claims abstract description 97
- 238000003062 neural network model Methods 0.000 claims abstract description 67
- 238000004891 communication Methods 0.000 claims abstract description 22
- 238000007781 pre-processing Methods 0.000 claims abstract description 13
- 238000000034 method Methods 0.000 claims description 40
- 239000013598 vector Substances 0.000 claims description 14
- 230000008569 process Effects 0.000 claims description 13
- 238000012545 processing Methods 0.000 claims description 9
- 238000013527 convolutional neural network Methods 0.000 claims description 5
- 230000009466 transformation Effects 0.000 claims description 5
- 238000010606 normalization Methods 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims 1
- 238000003703 image analysis method Methods 0.000 claims 1
- 238000013136 deep learning model Methods 0.000 abstract description 5
- 230000006870 function Effects 0.000 description 16
- 238000010586 diagram Methods 0.000 description 10
- 238000004590 computer program Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 6
- 230000035945 sensitivity Effects 0.000 description 6
- 238000001514 detection method Methods 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 238000010191 image analysis Methods 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 206010002091 Anaesthesia Diseases 0.000 description 1
- 230000037005 anaesthesia Effects 0.000 description 1
- 238000000137 annealing Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 238000001574 biopsy Methods 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 108090000623 proteins and genes Proteins 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5072—Grid computing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Artificial Intelligence (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The application belongs to the field of deep learning and intelligent analysis of medical images, in particular relates to an intelligent analysis system of medical images based on distributed deep learning, and aims to solve the problems of difficult model training, low classification precision and poor generalization capability caused by incapability of sharing medical data in the traditional intelligent analysis system of medical images. The system of the application comprises: the server is arranged at the service ends of the N medical centers and is connected with the center client through a communication link; the server comprises an image acquisition module which is configured to acquire medical images and perform image preprocessing; the central client includes a distributed model training and scheduling module configured to train the deep neural network models in a distributed manner and to centrally schedule each distributed deep neural network model for model iteration. The application avoids medical data sharing, simplifies the training difficulty of the multi-center model, and improves the analysis precision and generalization capability of the deep learning model.
Description
Technical Field
The application belongs to the field of deep learning and intelligent analysis of medical images, and particularly relates to an intelligent analysis system, method and equipment of medical images based on distributed deep learning.
Background
At present, the medical imaging technology is used for reflecting focus information of a patient and further realizing detection and classification, which is one of common and effective means, and can reduce the clinical limitations such as traumatism, operative anesthesia, time waiting and the like caused by the traditional means such as tissue biopsy or gene detection. However, analysis of medical images often relies on subjective experience of doctors, and a great deal of quantitative information about lesions contained in medical images is urgently needed to be developed. Therefore, the automatic and individual intelligent analysis of the medical images is matched with the current accurate medical concept, and the accuracy of medical image detection and classification tasks can be effectively improved, so that the efficiency of medical image analysis work is improved.
The deep learning technology has potential in tasks such as medical image detection and classification, and can create a feasible solution for solving the problems of automatic focus detection and classification. However, applying deep learning techniques in the medical field often requires a large amount of training data, and when differences between imaging phenotypes are small or there is large heterogeneity in the population, the sample size of the patient is often limited, which may result in poor generalization ability of the deep learning model. To address this challenge, data from multiple medical centers is typically collected to increase sample size and sample diversity, thereby enhancing the analysis effect of the deep learning model. On the other hand, however, the image data of the patient may occupy a large storage space, which is inconvenient to transmit; even if the transmission requirement is met, the sharing of the image data of the patient between the medical centers may involve legal and ethical issues such as privacy security, which also leads to the blockage of the analysis study of the medical images.
The combination of the distributed technology and the deep learning technology can provide a new opportunity for medical image analysis and research among a plurality of medical centers. The distributed deep learning method assumes that each medical center does not need to share original data and independently execute the deep learning training process, thereby having stronger safety and higher efficiency. However, the application of the current distributed deep learning method in medical image analysis is still limited to the theoretical demonstration and simulation stage, and the development of a distributed deep learning system with an asynchronous communication function and a distributed model training and scheduling function has important significance for the future intelligent, automatic and individual medical development direction. Aiming at the problems, the application provides a medical image intelligent analysis system based on distributed deep learning.
Disclosure of Invention
In order to solve the problems in the prior art, namely to solve the problems of difficult model training, low classification precision and poor generalization capability caused by the fact that the existing medical image intelligent analysis system cannot share medical data, the application provides a medical image intelligent analysis system based on distributed deep learning, which comprises a server side arranged in N medical centers and a center client side arranged in a remote server; each server is connected with the central client through a communication link:
the server comprises an image acquisition module; the image acquisition module is configured to acquire medical images to be analyzed and perform image preprocessing; associating the preprocessed medical images with each server as training sample data;
the center client comprises a distributed model training and scheduling module; the distributed model training and scheduling module is configured to train the deep neural network model in a distributed mode and intensively schedule each distributed deep neural network model by the central client to perform model iteration;
the server and the center client also comprise an image classification module; the image classification module is configured to acquire the category of the medical image to be analyzed through a trained deep neural network model;
the training method of the deep neural network model comprises the following steps of;
a10, the center client encodes the deep neural network model to be trained, the initialized model weight and the model training parameters into directional tensors, and respectively sends the directional tensors to the service ends of the N medical centers; the directivity tensor is a tensor of a set direction;
a20, each server trains the received deep neural network model based on the training sample data obtained locally, calculates a model training loss value, updates model weights, encodes the loss value and the model weights into directional vectors, and sends the directional vectors to the central client;
a30, the center client acquires the model training loss value and the updated model weight of each server and carries out weighted average treatment; taking the weighted average model weight as a new round of initialized model weight, and sending the model weight to each server again;
a40, the steps A20 and A30 are circulated until the deep neural network model converges.
In some preferred embodiments, the communication link connection between each server and the central client is implemented through a communication protocol and a chained model; the chain model is used for integrating a series of operations of the tensor into an operation chain, and each operation can represent the state or transformation of the tensor; the chain structure uses child attribute to access downwards at the head and parent attribute to access upwards, so that the asynchronous sending and receiving of the directional tensor between the server and the central client are realized.
In some preferred embodiments, the training sample data is associated with id index, ip address, port of each server using Torch Hook structure.
In some preferred embodiments, the image preprocessing includes image cropping, scaling, and normalization.
In some preferred embodiments, the deep neural network model is constructed based on densely connected convolutional neural networks; wherein, each layer is connected with all layers in front of the densely connected convolutional neural network in the channel dimension and is used as the input of the next layer.
In some preferred embodiments, the learning rate of the deep neural network model in the training process is calculated by the following method:
lr′＝max(lr′×0.95，lr 0 ×0.01)
or (b)
Wherein i represents the ith iteration round, r represents the total iteration round number, lr 0 Indicating the initial learning rate, lr' indicating the updated learning rate after the ith iteration.
In some preferred embodiments, the method for obtaining the model training loss value and the updated model weight of each server by the central client and performing weighted average processing includes:
wherein L is i，j Representing model training loss value, W calculated by ith iteration and jth server i，j Model weight, W representing the update of the jth server of the ith iteration i+1，0 The model weights after the weighted average processing are used.
In a second aspect of the present application, a medical image intelligent analysis method based on distributed deep learning is provided, the method comprising the following steps:
s10, the server side located in each medical center respectively acquires medical images to be analyzed and performs image preprocessing; associating the preprocessed medical images with each server as training sample data;
s20, training the deep neural network model in a distributed mode by a central client arranged on a remote server, and intensively scheduling each distributed deep neural network model by the central client to perform model iteration;
s30, the server and the central client acquire the category of the medical image to be analyzed through a trained deep neural network model;
the training method of the deep neural network model comprises the following steps:
a10, the center client encodes the deep neural network model to be trained, the initialized model weight and the model training parameters into directional tensors, and respectively sends the directional tensors to the service ends of the N medical centers; the directivity tensor is a tensor of a set direction;
a20, each server trains the received deep neural network model based on the training sample data obtained locally, calculates a model training loss value, updates model weights, encodes the loss value and the model weights into directional vectors, and sends the directional vectors to the central client;
a30, the center client acquires the model training loss value and the updated model weight of each server and carries out weighted average treatment; taking the weighted average model weight as a new round of initialized model weight, and sending the model weight to each server again;
a40, the steps A20 and A30 are circulated until the deep neural network model converges.
In a third aspect of the present application, a computer device is presented, comprising: at least one processor and a memory communicatively coupled to at least one of the processors; the memory stores instructions executable by the processor for execution by the processor to implement the distributed deep learning-based medical image intelligent analysis method.
In a fourth aspect of the present application, a computer readable storage medium is provided, where the computer readable storage medium stores computer instructions, where the computer instructions are configured to be executed by the computer to implement the distributed deep learning-based medical image intelligent analysis method.
The application has the beneficial effects that:
the application avoids the transmission and sharing of medical data, simplifies the training difficulty of the multi-center model, and improves the analysis precision of medical images and the generalization capability of the deep learning model.
1) The application is based on the flexibility and expandability of the existing distributed technology, and effectively combines a plurality of single machine resources under the condition of avoiding data sharing. The training data are respectively delivered to the service end nodes positioned in each medical center for training, and the distributed training of the deep neural network model can be completed only by transmitting the model and parameters between each service end and the central client, so that the difficulty of model training is simplified, the analysis precision of medical images and the generalization capability of the deep learning model are improved, and the method is an effective strategy for analyzing a large-scale medical image data set. Meanwhile, the problem that medical data cannot be shared due to privacy security of the medical data is effectively avoided, and the security is higher.
2) The application effectively combines the PyTorch deep learning framework, the PySyft distributed computing framework and the WebSocket communication technology, realizes the distributed computing function based on chain tensor operation and the asynchronous communication function between each medical center server and each medical center client, and provides a distributed deep learning scheme with simple interface and strong expandability for intelligent analysis of medical images.
3) In addition, the intelligent analysis of medical images by using distributed deep learning has important significance for future intelligent, automatic and individual medical development directions.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the detailed description of non-limiting embodiments made with reference to the following drawings.
FIG. 1 is a schematic diagram of a distributed deep learning-based medical image intelligent analysis system according to an embodiment of the present application;
FIG. 2 is a schematic flow diagram of server side image acquisition and processing according to an embodiment of the present application;
FIG. 3 is an exemplary view of a acquired medical image in accordance with one embodiment of the present application;
FIG. 4 is a schematic diagram of training and iterating a deep neural network model according to one embodiment of the present application;
FIG. 5 is a schematic diagram of the architecture of deep neural network model training and iteration of one embodiment of the present application;
fig. 6 is a schematic diagram of a computer system suitable for use in implementing an embodiment of the application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The application is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the application and are not limiting of the application. It should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be combined with each other.
The application relates to a medical image intelligent analysis system based on distributed deep learning, which is shown in fig. 1, and comprises a server side arranged in N medical centers and a center client side arranged in a remote server; each server is connected with the central client through a communication link:
the server comprises an image acquisition module; the image acquisition module is configured to acquire medical images to be analyzed and perform image preprocessing; associating the preprocessed medical images with each server as training sample data;
the center client comprises a distributed model training and scheduling module; the distributed model training and scheduling module is configured to train the deep neural network model in a distributed mode and intensively schedule each distributed deep neural network model by the central client to perform model iteration;
the server and the center client also comprise an image classification module; the image classification module is configured to acquire the category of the medical image to be analyzed through a trained deep neural network model;
the training method of the deep neural network model comprises the following steps of;
a10, the center client encodes the deep neural network model to be trained, the initialized model weight and the model training parameters into directional tensors, and respectively sends the directional tensors to the service ends of the N medical centers; the directivity tensor is a tensor of a set direction;
a20, each server trains the received deep neural network model based on the training sample data obtained locally, calculates a model training loss value, updates model weights, encodes the loss value and the model weights into directional vectors, and sends the directional vectors to the central client;
a30, the center client acquires the model training loss value and the updated model weight of each server and carries out weighted average treatment; taking the weighted average model weight as a new round of initialized model weight, and sending the model weight to each server again;
a40, the steps A20 and A30 are circulated until the deep neural network model converges.
In order to more clearly describe the distributed deep learning-based medical image intelligent analysis system of the application, each step in one embodiment of the method of the application is described in detail below.
In the following embodiments, a method for acquiring a medical image in a distributed manner is described, a method for training and iterating a deep neural network model in a distributed manner is described, and a specific process for acquiring an analysis result of a medical image by a medical image intelligent analysis system based on distributed deep learning is described in detail.
1. Distributed acquisition of medical images, as shown in FIG. 2
Medical images are respectively acquired at the service ends of the N medical centers, training sample data are packaged into a Dataset class based on a PyTorch API, and the DataLoader is transmitted; performing image preprocessing on training sample data; based on PySyft API, training sample data are further packaged into BaseDateset base class and are respectively associated with each medical center service end;
in this embodiment, medical images are independently acquired at N medical center servers, including modalities such as CT, MRI, and X-ray, as shown in fig. 3, and stored in DICOM format or in a common picture format.
The medical image is subjected to image preprocessing, including image cutting, scaling and standardization, and specifically comprises the following steps: firstly, cutting out an image of a medical image, and scaling the cut medical image to a preset size; secondly, converting the medical image into a tensor form; finally, the medical image is standardized, and the standardization can be realized by adopting methods such as z fraction or Min-Max standardization.
Defining index ids of N medical center service ends, and acquiring an ip address and a port; and loading training sample data at the local ip address of each server and a preset port, packaging the training sample data by adopting a BaseDateset base class provided by a PySyft framework, and associating the training sample data with the server id index, the ip address and the port by adopting a Torch Hook function.
2. Training and iterating the deep neural network model, as shown in FIG. 4
A10, the center client encodes the deep neural network model to be trained, the initialized model weight and the model training parameters into directional tensors, and respectively sends the directional tensors to the service ends of the N medical centers; the directivity tensor is a tensor of a set direction;
in this embodiment, a deep neural network model is written at the central client based on the PyTorch, and its structure is specifically designed as a convolutional neural network adopting a dense connection mechanism, for example: denseNet-64, denseNet-121, etc., each layer accepts all of its previous layers as its additional input, i.e., each layer will be connected together in the channel dimension with all of its previous layers and as input to the next layer; the nonlinear transformation functions used include operations such as batch normalization, reLU, and convolution.
Randomly initializing model weight as W 0，0 Meanwhile, setting distributed training parameters for the model, wherein the distributed training parameters comprise iteration round number, batch size, learning rate, loss function and optimizer, the loss function can adopt cross entropy loss function, log likelihood cost function and the like for classifying tasks, can also adopt L1 loss, L2 loss and the like for detecting tasks, and can also adopt Dice loss and the like for dividing tasks; the optimizer can adopt a random gradient descent optimizer, a self-adaptive moment estimation optimizer and the like; the setting strategy of the learning rate can be optionally one of the following: constant learning rate lr 0 Performing learning rate decay according to formula (1), performing learning rate cosine annealing operation according to formula (2), wherein i represents the ith iteration round, r represents the total iteration round number, and lr' represents the updated learning rate after the ith iteration round:
lr′＝max(lr′×0.95，lr 0 ×0.01) (1)
the server and the center client can establish persistent connection by only completing one handshake through the WebSocket API, and perform bidirectional byte transmission. In this embodiment, an asynchronous communication link connection between a central client and N servers is established through WebSocket communication protocol of an asynchronous communication module, and id indexes, ip addresses and port numbers corresponding to the servers are obtained, and deep neural network models, weights and training parameters of the central client are encoded into directional tensors (pointensors); the directivity tensor is then sent from the central client to each server through the chain model (SyftTensor) of the chain coding module. Namely, the communication link connection in the application comprises a chain coding module and an asynchronous communication module; the chain model in the chain coding module is used for integrating a series of operations of the tensor into an operation chain, and each operation can represent the state or transformation of the tensor; the chain structure uses child attribute to access downwards at the head and parent attribute to access upwards, so that the asynchronous sending and receiving of the directional tensor between the server and the central client are realized.
A20, each server trains the received deep neural network model based on the training sample data obtained locally, calculates a model training loss value, updates model weights, encodes the loss value and the model weights into directional vectors, and sends the directional vectors to the central client;
in this embodiment, N servers respectively receive directional tensors of the deep neural network model, the weights and the training parameters through a chain model (SyftTensor), and perform independent training on the training sample data loaded locally. Calculating a model loss value L of the ith round of iteration by combining class labels of training sample data i，1 ，L i，2 ，L i，3 ....L i，N And updating the model weight of the ith round to W by counter-propagating the calculated gradient i，1 ，W i，2 ，W i，3 ....W i，N . And encoding the model loss value and the model updated weight into a directivity tensor, and transmitting the directivity tensor to the center client.
A30, the center client acquires the model training loss value and the updated model weight of each server and carries out weighted average treatment; taking the weighted average model weight as a new round of initialized model weight, and sending the model weight to each server again;
in this embodiment, the central client receives the model loss values and updated model weights from the N servers, and performs federal computation, that is, centralized scheduling, on the distributed model, as shown in fig. 5, an average value of the model weights may be calculated according to equation (3) or a weighted average value of the model weights may be calculated according to equation (4):
and taking the weighted average model weight as the initial model weight of a new iteration and sending the initial model weight to N servers.
A40, the steps A20 and A30 are circulated until the deep neural network model converges.
In this embodiment, the steps a20 and a30 are repeated until the model training termination condition is satisfied, and the optimal deep neural network model is obtained. The model training termination conditions were: the actual training iteration number of each server reaches the preset distributed training iteration number, or after each server receives the updated model weight, the model precision reaches the expected requirement. And then, the central client transmits the optimal deep neural network model to N servers, and the efficiency of the model on multi-center data is calculated respectively, wherein the efficiency comprises evaluation indexes such as AUC, accuracy, sensitivity and specificity.
3. Medical image intelligent analysis system based on distributed deep learning acquires analysis results of medical images
The medical center server side of the medical image intelligent analysis system based on the distributed deep learning comprises an image acquisition module; the image acquisition module is configured to acquire a medical image to be analyzed and perform image preprocessing; associating the preprocessed medical images with each server as training sample data;
the central client comprises a distributed model training and scheduling module; the distributed model training and scheduling module is configured to train the deep neural network model in a distributed mode and intensively schedule each distributed deep neural network model by the central client to perform model iteration;
the server side and the center client side also comprise image classification modules; the image classification module is configured to acquire an analysis result of the medical image to be analyzed, namely the category of the medical image through the trained deep neural network model;
in the embodiment, a medical image to be analyzed is firstly obtained and is used as a model input; then, the deep neural network model is trained and iterated in a distributed mode; and finally, obtaining the category of the medical image to be analyzed through the trained deep neural network model.
In addition, in the present application, each parameter is preferably set as:
and (3) loading data: 55000 training samples, 10000 test samples, 5 medical centers' service terminals, 1 center client;
data preprocessing: random resize dcrop, zoom the picture into 64 x 64 size, the picture carries on the standardization process (meet the mean value as 0, standard deviation as 1);
deep neural network model: denseNet-121;
distributed training parameters: iteration 100 rounds, batch size 32, constant learning rate 0.01, cross entropy loss function, random gradient descent optimizer;
the test model is time-consuming: the CPU is adopted for model training, and the time for completing one round of model training and iteration is about 5 minutes;
test model efficacy: AUC of the server 1 reaches 0.87, accuracy is 0.78, sensitivity is 0.83, and specificity is 0.76; the AUC of the server 2 reaches 0.87, the accuracy is 0.76, the sensitivity is 0.86, and the specificity is 0.73; the AUC of the server 3 reaches 0.87, the accuracy is 0.76, the sensitivity is 0.86, and the specificity is 0.73; the AUC of the server 4 reaches 0.87, the accuracy is 0.76, the sensitivity is 0.86, and the specificity is 0.73; the AUC of the server 5 reaches 0.87, the accuracy is 0.74, the sensitivity is 0.86, and the specificity is 0.71.
It should be noted that, in the distributed deep learning-based medical image intelligent analysis system provided in the foregoing embodiment, only the division of the foregoing functional modules is illustrated, and in practical application, the foregoing functional allocation may be performed by different functional modules according to needs, that is, the modules or steps in the foregoing embodiment of the present application are further decomposed or combined, for example, the modules in the foregoing embodiment may be combined into one module, or may be further split into multiple sub-modules, so as to complete all or part of the functions described above. The names of the modules and steps related to the embodiments of the present application are merely for distinguishing the respective modules or steps, and are not to be construed as unduly limiting the present application.
The medical image intelligent analysis method based on distributed deep learning in the second embodiment of the application comprises the following steps:
s10, the server side located in each medical center respectively acquires medical images to be analyzed and performs image preprocessing; associating the preprocessed medical images with each server as training sample data;
s20, training the deep neural network model in a distributed mode by a central client arranged on a remote server, and intensively scheduling each distributed deep neural network model by the central client to perform model iteration;
s30, the server and the central client acquire the category of the medical image to be analyzed through a trained deep neural network model;
the training method of the deep neural network model comprises the following steps:
a10, the center client encodes the deep neural network model to be trained, the initialized model weight and the model training parameters into directional tensors, and respectively sends the directional tensors to the service ends of the N medical centers; the directivity tensor is a tensor of a set direction;
a20, each server trains the received deep neural network model based on the training sample data obtained locally, calculates a model training loss value, updates model weights, encodes the loss value and the model weights into directional vectors, and sends the directional vectors to the central client;
a30, the center client acquires the model training loss value and the updated model weight of each server and carries out weighted average treatment; taking the weighted average model weight as a new round of initialized model weight, and sending the model weight to each server again;
a40, the steps A20 and A30 are circulated until the deep neural network model converges.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working processes and related descriptions of the above-described method may refer to corresponding processes in the foregoing system embodiments, which are not repeated herein.
A third embodiment of the present application proposes a computer device comprising: at least one processor and a memory communicatively coupled to at least one of the processors; the memory stores instructions executable by the processor for execution by the processor to implement the distributed deep learning-based medical image intelligent analysis method.
The fourth embodiment of the present application provides a computer readable storage medium, which is characterized in that the computer readable storage medium stores computer instructions, and the computer instructions are used for being executed by the computer to implement the distributed deep learning-based medical image intelligent analysis method.
It will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process of the storage device and the processing device described above and the related description may refer to the corresponding process in the foregoing method example, which is not repeated herein.
Referring now to FIG. 6, there is shown a schematic diagram of a computer system suitable for use with servers implementing embodiments of the present systems, methods, and apparatus. The server illustrated in fig. 6 is merely an example, and should not be construed as limiting the functionality and scope of use of embodiments of the present application.
As shown in fig. 6, the computer system includes a central processing unit (CPU, central Processing Unit) 601, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage section 608 into a random access Memory (RAM, random Access Memory) 603. In the RAM603, various programs and data required for system operation are also stored. The CPU601, ROM602, and RAM603 are connected to each other through a bus 604. An Input/Output (I/O) interface 605 is also connected to bus 604.
The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, mouse, etc.; an output portion 607 including a cathode ray tube, a liquid crystal display, and the like, a speaker, and the like; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a lan card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The drive 610 is also connected to the I/O interface 605 as needed. Removable media 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed as needed on drive 610 so that a computer program read therefrom is installed as needed into storage section 608.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 609 and/or installed from the removable medium 611. The above-described functions defined in the method of the present application are performed when the computer program is executed by the CPU 601. The computer readable medium of the present application may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium may be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a RAM, a ROM, an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present application, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, and the like, or any suitable combination of the foregoing.
Computer program code for carrying out operations of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the C-programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of remote computers, the remote computer may be connected to the user's computer through any kind of network, including a local area network or a wide area network, or may be connected to an external computer (e.g., connected through the internet using an internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The terms "first," "second," and the like, are used for distinguishing between similar objects and not for describing a particular sequential or chronological order.
The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus/apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus/apparatus.
Thus far, the technical solution of the present application has been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of protection of the present application is not limited to these specific embodiments. Equivalent modifications and substitutions for related technical features may be made by those skilled in the art without departing from the principles of the present application, and such modifications and substitutions will fall within the scope of the present application.
Claims (9)
1. The medical image intelligent analysis system based on the distributed deep learning is characterized by comprising a service end arranged in N medical centers and a center client end arranged in a remote server; each server is connected with the central client through a communication link;
the server comprises an image acquisition module; the image acquisition module is configured to acquire medical images to be analyzed and perform image preprocessing; associating the preprocessed medical images with each server as training sample data;
the center client comprises a distributed model training and scheduling module; the distributed model training and scheduling module is configured to train the deep neural network model in a distributed mode and intensively schedule each distributed deep neural network model by the central client to perform model iteration;
the server side and the center client side also comprise an image classification module; the image classification module is configured to acquire the category of the medical image to be analyzed through a trained deep neural network model;
the training method of the deep neural network model comprises the following steps:
a10, the center client encodes the deep neural network model to be trained, the initialized model weight and the model training parameters into directional tensors, and sends the directional tensors to the service ends of the N medical centers respectively; the directivity tensor is a tensor of a set direction;
a20, each server trains the received deep neural network model based on the training sample data obtained locally, calculates a model training loss value, updates model weights, encodes the loss value and the model weights into directional vectors, and sends the directional vectors to the central client;
a30, the center client acquires the model training loss value and the updated model weight of each server and carries out weighted average treatment; taking the weighted average model weight as a new round of initialized model weight, and sending the model weight to each server again;
a40, circulating the steps A20 and A30 until the deep neural network model converges;
the communication link connection between each server and the central client is realized through a WebSocket communication protocol and a chain model; the chain model is used for integrating a series of operations of the tensor into an operation chain, and each operation can represent the state or transformation of the tensor; the chain structure uses child attribute to access downwards at the head and parent attribute to access upwards, so that the asynchronous sending and receiving of the directional tensor between the server and the central client are realized.
2. The distributed deep learning based medical image intelligent analysis system according to claim 1, wherein the training sample data is associated with id index, ip address and port of each server by using Torch Hook structure.
3. The distributed deep learning based medical image intelligent analysis system of claim 1, wherein the image preprocessing includes image cropping, scaling and normalization.
4. The distributed deep learning-based medical image intelligent analysis system of claim 1, wherein the deep neural network model is constructed based on a densely connected convolutional neural network; wherein, each layer is connected with all layers in front of the densely connected convolutional neural network in the channel dimension and is used as the input of the next layer.
5. The intelligent analysis system of medical images based on distributed deep learning according to claim 1, wherein the calculation method of the learning rate of the deep neural network model in the training process is as follows:
lr′＝max(lr′×0.95,lr 0 ×0.01)
or (b)
Wherein i represents the ith iteration round, r represents the total iteration round number, lr 0 Indicating the initial learning rate, lr' indicating the updated learning rate after the ith iteration.
6. The intelligent analysis system of medical images based on distributed deep learning according to claim 1, wherein the central client obtains model training loss values and updated model weights of each server and performs weighted average processing, and the method comprises the following steps:
wherein L is i,j Representing model training loss value, W calculated by ith iteration and jth server i,j Model weight, W representing the update of the jth server of the ith iteration i+1,0 The model weights after the weighted average processing are used.
7. The intelligent medical image analysis method based on distributed deep learning is characterized by comprising the following steps of:
s10, the server side located in each medical center respectively acquires medical images to be analyzed and performs image preprocessing; associating the preprocessed medical images with each server as training sample data;
s20, training the deep neural network model in a distributed mode by a central client arranged on a remote server, and intensively scheduling each distributed deep neural network model by the central client to perform model iteration;
s30, the server side and the central client side acquire the category of the medical image to be analyzed through a trained deep neural network model;
the training method of the deep neural network model comprises the following steps:
a10, the center client encodes the deep neural network model to be trained, the initialized model weight and the model training parameters into directional tensors, and sends the directional tensors to the service ends of the N medical centers respectively; the directivity tensor is a tensor of a set direction;
a20, each server trains the received deep neural network model based on the training sample data obtained locally, calculates a model training loss value, updates model weights, encodes the loss value and the model weights into directional vectors, and sends the directional vectors to the central client;
a30, the center client acquires the model training loss value and the updated model weight of each server and carries out weighted average treatment; taking the weighted average model weight as a new round of initialized model weight, and sending the model weight to each server again;
a40, circulating the steps A20 and A30 until the deep neural network model converges;
the communication link connection between each server and the central client is realized through a WebSocket communication protocol and a chain model; the chain model is used for integrating a series of operations of the tensor into an operation chain, and each operation can represent the state or transformation of the tensor; the chain structure uses child attribute to access downwards at the head and parent attribute to access upwards, so that the asynchronous sending and receiving of the directional tensor between the server and the central client are realized.
8. A computer device, comprising: at least one processor and a memory communicatively coupled to at least one of the processors; wherein the memory stores instructions executable by the processor for execution by the processor to implement the distributed deep learning based medical image intelligent analysis method of claim 7.
9. A computer-readable storage medium storing computer instructions for execution by the computer to implement the distributed deep learning-based medical image intelligent analysis method of claim 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110268700.9A CN112988382B (en) | 2021-03-12 | 2021-03-12 | Medical image intelligent analysis system based on distributed deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110268700.9A CN112988382B (en) | 2021-03-12 | 2021-03-12 | Medical image intelligent analysis system based on distributed deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112988382A CN112988382A (en) | 2021-06-18 |
| CN112988382B true CN112988382B (en) | 2023-09-19 |
Family
ID=76335066
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110268700.9A Active CN112988382B (en) | 2021-03-12 | 2021-03-12 | Medical image intelligent analysis system based on distributed deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112988382B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114140478B (en) * | 2022-01-30 | 2022-06-03 | 电子科技大学 | Federal learning method, system, device and medium for medical image segmentation |
| CN114463330B (en) * | 2022-04-12 | 2022-07-01 | 江苏康医通科技有限公司 | CT data collection system, method and storage medium |
| CN115115064B (en) * | 2022-07-11 | 2023-09-05 | 山东大学 | Semi-asynchronous federal learning method and system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106297774A (en) * | 2015-05-29 | 2017-01-04 | 中国科学院声学研究所 | The distributed parallel training method of a kind of neutral net acoustic model and system |
| CN106878304A (en) * | 2017-02-15 | 2017-06-20 | 国网天津市电力公司 | A kind of method of the link multiplexing of distributed agent |
| CN109032671A (en) * | 2018-06-25 | 2018-12-18 | 电子科技大学 | A kind of distributed deep learning method and system based on data parallel strategy |
| CN110956202A (en) * | 2019-11-13 | 2020-04-03 | 重庆大学 | Image training method, system, medium and intelligent device based on distributed learning |
| CN111444019A (en) * | 2020-03-31 | 2020-07-24 | 中国科学院自动化研究所 | Cloud-end-collaborative deep learning model distributed training method and system |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11195057B2 (en) * | 2014-03-18 | 2021-12-07 | Z Advanced Computing, Inc. | System and method for extremely efficient image and pattern recognition and artificial intelligence platform |
| US11373115B2 (en) * | 2018-04-09 | 2022-06-28 | Here Global B.V. | Asynchronous parameter aggregation for machine learning |
| US11315013B2 (en) * | 2018-04-23 | 2022-04-26 | EMC IP Holding Company LLC | Implementing parameter server in networking infrastructure for high-performance computing |
-
2021
- 2021-03-12 CN CN202110268700.9A patent/CN112988382B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106297774A (en) * | 2015-05-29 | 2017-01-04 | 中国科学院声学研究所 | The distributed parallel training method of a kind of neutral net acoustic model and system |
| CN106878304A (en) * | 2017-02-15 | 2017-06-20 | 国网天津市电力公司 | A kind of method of the link multiplexing of distributed agent |
| CN109032671A (en) * | 2018-06-25 | 2018-12-18 | 电子科技大学 | A kind of distributed deep learning method and system based on data parallel strategy |
| CN110956202A (en) * | 2019-11-13 | 2020-04-03 | 重庆大学 | Image training method, system, medium and intelligent device based on distributed learning |
| CN111444019A (en) * | 2020-03-31 | 2020-07-24 | 中国科学院自动化研究所 | Cloud-end-collaborative deep learning model distributed training method and system |
Non-Patent Citations (5)
| Title |
|---|
| Sergeev, Alexander, and Mike Del Balso.."Horovod: fast and easy distributed deep learning in TensorFlow".《arXiv preprint arXiv:1802.05799 (2018)》.2018,第1-10页. * |
| Turina, Valeria, et al.."Combining split and federated architectures for efficiency and privacy in deep learning".《Proceedings of the 16th International Conference on emerging Networking EXperiments and Technologies. 2020》.2020,第562-563页. * |
| Vepakomma, Praneeth, et al.."NoPeek: Information leakage reduction to share activations in distributed deep learning" .《2020 International Conference on Data Mining Workshops (ICDMW)》.2020,第933-942页. * |
| Zhang, Liwen , et al. ."A Deep Learning Risk Prediction Model for Overall Survival in Patients with Gastric Cancer: A Multicenter Study".《 Radiotherapy and Oncology》.2020,第73-80页. * |
| 谭作文 ; 张连福 ; ."机器学习隐私保护研究综述".《软件学报》.2020,第2128-2150页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112988382A (en) | 2021-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112988382B (en) | Medical image intelligent analysis system based on distributed deep learning | |
| CN108427939B (en) | Model generation method and device | |
| CN109241141B (en) | Deep learning training data processing method and device | |
| US20190096060A1 (en) | Method and apparatus for annotating medical image | |
| CN107622240B (en) | Face detection method and device | |
| CN108197652B (en) | Method and apparatus for generating information | |
| CN109002842A (en) | Image-recognizing method and device | |
| CN109698022A (en) | Medical image data utilization and sharing platform | |
| CN110517254A (en) | The automatic delineation method in clinical target area, device and relevant device based on deep learning | |
| CN111243711B (en) | Feature recognition in medical imaging | |
| CN109243600B (en) | Method and apparatus for outputting information | |
| CN111274490A (en) | Method and device for processing consultation information | |
| CN111985197A (en) | Template generation method based on medical information | |
| CN112349428A (en) | Family health code query method and equipment | |
| CN112925785A (en) | Data cleaning method and device | |
| CN111523593A (en) | Method and apparatus for analyzing medical images | |
| CN114627331A (en) | Model training method and device | |
| CN114358316A (en) | Federal learning system and large-scale image training method and device thereof | |
| CN112397194B (en) | Method, device and electronic equipment for generating patient disease attribution interpretation model | |
| CN112434323A (en) | Model parameter obtaining method and device, computer equipment and storage medium | |
| CN111833993B (en) | AI-based regional image remote quality control management system | |
| CN113255819A (en) | Method and apparatus for identifying information | |
| CN109408716B (en) | Method and device for pushing information | |
| CN112446192A (en) | Method, device, electronic equipment and medium for generating text labeling model | |
| CN117174319B (en) | Sepsis time sequence prediction method and system based on knowledge graph |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |