CN113255889A - Occupational pneumoconiosis multi-modal analysis method based on deep learning - Google Patents
Occupational pneumoconiosis multi-modal analysis method based on deep learning Download PDFInfo
- Publication number
- CN113255889A CN113255889A CN202110579022.8A CN202110579022A CN113255889A CN 113255889 A CN113255889 A CN 113255889A CN 202110579022 A CN202110579022 A CN 202110579022A CN 113255889 A CN113255889 A CN 113255889A
- Authority
- CN
- China
- Prior art keywords
- cnn
- pneumoconiosis
- frog
- analysis
- fitness
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 206010035653 pneumoconiosis Diseases 0.000 title claims abstract description 64
- 238000004458 analytical method Methods 0.000 title claims abstract description 53
- 238000013135 deep learning Methods 0.000 title claims abstract description 16
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 76
- 238000000034 method Methods 0.000 claims abstract description 23
- 238000011976 chest X-ray Methods 0.000 claims abstract description 19
- 230000006870 function Effects 0.000 claims abstract description 15
- 238000012545 processing Methods 0.000 claims abstract description 12
- 108090000623 proteins and genes Proteins 0.000 claims description 44
- 238000011176 pooling Methods 0.000 claims description 18
- 230000008569 process Effects 0.000 claims description 11
- 238000003745 diagnosis Methods 0.000 claims description 10
- 241000269350 Anura Species 0.000 claims description 9
- 230000009191 jumping Effects 0.000 claims description 9
- 230000004913 activation Effects 0.000 claims description 8
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 230000002950 deficient Effects 0.000 claims description 3
- 238000002156 mixing Methods 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims description 3
- 238000000638 solvent extraction Methods 0.000 claims description 2
- 230000035622 drinking Effects 0.000 claims 1
- 230000000391 smoking effect Effects 0.000 claims 1
- 210000004072 lung Anatomy 0.000 abstract description 9
- 230000036541 health Effects 0.000 abstract description 7
- 238000010223 real-time analysis Methods 0.000 abstract description 2
- 238000011897 real-time detection Methods 0.000 abstract description 2
- 210000000038 chest Anatomy 0.000 description 7
- 239000000428 dust Substances 0.000 description 7
- 238000002601 radiography Methods 0.000 description 6
- 239000003245 coal Substances 0.000 description 5
- 201000010099 disease Diseases 0.000 description 4
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 4
- 238000013473 artificial intelligence Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 239000011435 rock Substances 0.000 description 3
- 206010008479 Chest Pain Diseases 0.000 description 2
- 206010020772 Hypertension Diseases 0.000 description 2
- 241000208125 Nicotiana Species 0.000 description 2
- 235000002637 Nicotiana tabacum Nutrition 0.000 description 2
- VYPSYNLAJGMNEJ-UHFFFAOYSA-N Silicium dioxide Chemical compound O=[Si]=O VYPSYNLAJGMNEJ-UHFFFAOYSA-N 0.000 description 2
- 210000001132 alveolar macrophage Anatomy 0.000 description 2
- 238000005422 blasting Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 206010012601 diabetes mellitus Diseases 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000003902 lesion Effects 0.000 description 2
- 230000001681 protective effect Effects 0.000 description 2
- 208000024891 symptom Diseases 0.000 description 2
- 201000008827 tuberculosis Diseases 0.000 description 2
- 206010006322 Breath holding Diseases 0.000 description 1
- 206010011224 Cough Diseases 0.000 description 1
- 206010016654 Fibrosis Diseases 0.000 description 1
- 208000028571 Occupational disease Diseases 0.000 description 1
- 208000002193 Pain Diseases 0.000 description 1
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000003123 bronchiole Anatomy 0.000 description 1
- 239000004566 building material Substances 0.000 description 1
- 239000002817 coal dust Substances 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000004761 fibrosis Effects 0.000 description 1
- 230000003862 health status Effects 0.000 description 1
- 230000004199 lung function Effects 0.000 description 1
- 210000002540 macrophage Anatomy 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000001717 pathogenic effect Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 210000001991 scapula Anatomy 0.000 description 1
- 239000000377 silicon dioxide Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 210000000115 thoracic cavity Anatomy 0.000 description 1
- 210000000779 thoracic wall Anatomy 0.000 description 1
- 210000001519 tissue Anatomy 0.000 description 1
- 230000005641 tunneling Effects 0.000 description 1
- 238000005406 washing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10116—X-ray image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30061—Lung
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a deep learning-based occupational pneumoconiosis multi-modal analysis method, which belongs to the field of pneumoconiosis analysis and comprises the following steps: collecting chest X-ray image information and personal basic information of a person; carrying out word vectorization processing on the personal basic information; constructing a one-dimensional convolutional neural network and a two-dimensional convolutional neural network, and establishing a multi-mode convolutional neural network MM-CNN model on the basis; the two kinds of information are used as the input of a multi-modal convolutional neural network MM-CNN model, a multi-classification MM-CNN pneumoconiosis analysis model is established, and an objective function is formed under the condition that corresponding constraints are met; optimizing the hyper-parameters of the multi-classification MM-CNN pneumoconiosis analysis model by adopting a mixed frog-jumping algorithm SFLA; and analyzing the chest X-ray image information of the person and the personal information after word vectorization by adopting the optimized multi-classification MM-CNN pneumoconiosis analysis model, and outputting an analysis result. The method can realize accurate and real-time detection and analysis of the lung health of the personnel, and complete early warning of part of occupational pneumoconiosis.
Description
Technical Field
The invention belongs to the field of pneumoconiosis analysis, and particularly relates to a deep learning-based occupational pneumoconiosis multi-modal analysis method.
Background
In recent years, the continuous stable improvement of the safety production in China is realized, the rapid reduction of the number of production safety accidents and death people for years is realized, but the situation of occupational health work is still very severe. Occupational pneumoconiosis is one of occupational diseases and is mainly distributed in coal, colored, mechanical, building material, light industry and other industrial industries. For example, in the coal mine production process, many dust (mainly including coal dust and silicon dust) is generated in many links such as rock tunnel blasting, rock tunnel loading, rock tunnel tunneling, coal tunnel blasting, coal tunnel reinforcement, coal preparation transportation, and the like, and the excessive dust is a fierce cause of pneumoconiosis. Pneumoconiosis is a disease in which lung tissue fibrosis, which occurs as a result of workers inhaling large quantities of free silica and other dust for a long period of time during industrial activities, predominates. Most of the dust is discharged, but a part of the dust is retained in bronchioles and alveoli for a long time and is continuously phagocytosed by alveolar macrophages, and the dust-swallowed macrophages are the main pathogenic factors. A series of studies have shown that after a pneumoconiosis lesion is formed, the residual dust in the lung continues to react with alveolar macrophages, which is the main reason why the lesion continues to develop even though the pneumoconiosis patient is out of dust operation. The common symptoms of patients with pneumoconiosis are chest distress, chest pain, short breath, cough, general weakness, serious patients losing labor capacity even can not lie flat, and finally lung function failure, kneeling and death are caused, and the symptoms of the patients are not witnessed.
Pneumoconiosis is an incurable disease, no specific medicine for curing pneumoconiosis exists in the world at present, and lung washing can only relieve the pain of patients to a certain extent and slow down the development of the disease, but cannot reverse the disease fundamentally. And the early detection aiming at the lung health condition of the personnel can realize the early recognition of the occupational pneumoconiosis, improve the early discovery rate of the occupational pneumoconiosis and reduce the serious burden of the personnel caused by the occupational pneumoconiosis. Therefore, it is necessary and important to develop a diagnosis apparatus and an analysis method for occupational pneumoconiosis of this special group, which can realize accurate and real-time measurement and analysis of the lung health status of the personnel, and have important significance for early warning of some occupational pneumoconiosis and ensuring the life health of the personnel.
At present, the diagnosis of occupational pneumoconiosis is mainly determined by 'diagnosis of occupational pneumoconiosis' (GBZ70-2015), and doctors rely on personal experience to perform diagnosis and analysis of pneumoconiosis based on relevant diagnosis standards and principles by comparing and analyzing chest slices of patients and X-ray diagnosis standard slices of pneumoconiosis. With the continuous development and application of the artificial intelligence algorithm, the application of the artificial intelligence algorithm to the auxiliary analysis of the chest radiography image data becomes a research hotspot and a series of successful applications are achieved. However, the chest radiography acquisition process is influenced by various factors such as the position of the irradiated object, exposure conditions, operation errors, films and the like, so that the chest radiography has uneven image quality levels and high difficulty in feature extraction and analysis, and the defects of low identification precision, long time consumption and the like exist when the traditional artificial intelligence algorithm is directly used for analyzing the chest radiography. With the proposal of deep learning theory and the improvement of numerical computing equipment, Convolutional Neural Network (CNN) has been rapidly developed, and CNN is used as one of the best modes for feature extraction in image processing and text analysis and is applied to the fields of computer vision, natural language processing and the like. Although the CNN model has excellent performance, how to design a suitable network structure for a specific application problem and find the optimal model parameters is a difficult problem in the CNN application process.
In view of the above, the invention provides a deep learning-based multi-modal analysis method for occupational pneumoconiosis.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides a deep learning-based occupational pneumoconiosis multimodal analysis method.
In order to achieve the above purpose, the invention provides the following technical scheme:
a multi-modal analysis method for occupational pneumoconiosis based on deep learning comprises the following steps:
collecting chest X-ray image information and personal basic information of a person;
performing word vectorization processing on the personal basic information;
constructing a One-dimensional Convolutional Neural Network (1D-CNN) and a Two-dimensional Convolutional Neural Network (2D-CNN), and establishing a Multi-modal Convolutional Neural Network (MM-CNN) model on the basis;
taking chest X-ray image information of personnel and personal information after word vectorization as input of a multi-mode convolutional neural network MM-CNN model, establishing a multi-classification MM-CNN pneumoconiosis analysis model for occupational pneumoconiosis analysis, and forming an objective function under the condition of meeting corresponding constraints;
adopting a mixed Frog-jumping algorithm (SHuffled Frog leapingAlgorithm, SFLA) to optimize the hyper-parameters of the multi-classification MM-CNN pneumoconiosis analysis model;
and analyzing the chest X-ray image information and the personal information after word vectorization processing of the personnel by adopting the optimized multi-classification MM-CNN pneumoconiosis analysis model, and outputting the result of the occupational pneumoconiosis analysis.
Preferably, in the word vectorization processing of the personal basic information, a skip-gram model in word2vec is adopted to perform word vectorization conversion of the personal basic information of the person, the size of a context window is set to be 10, the size of a word vector dimension is set to be 50, and the sampling size is set to be 1 e-3.
Preferably, the building of the one-dimensional convolutional neural network 1D-CNN is specifically that the number of layers of the one-dimensional convolutional neural network is deepened by connecting a plurality of one-dimensional convolutional units, the sizes of convolutional kernels are all set to 3, when one convolutional layer is the lth convolutional layer, the number of convolutional kernels is 8 × L, the one-dimensional pooling layer adopts a maximum pooling mode, and the pooling size is 2;
the method for constructing the two-dimensional convolutional neural network specifically comprises the steps of deepening the layer number of the two-dimensional convolutional neural network by connecting a plurality of two-dimensional convolutional units, setting the sizes of convolutional kernels to be 3 multiplied by 3, and when one convolutional layer is the L-th convolutional layer, the number of the convolutional kernels is 8 multiplied by L, wherein the two-dimensional pooling layer adopts a maximum pooling mode, and the pooling size is 2 multiplied by 2.
Preferably, the establishing of the multi-classification MM-CNN model for the analysis and diagnosis of the occupational pneumoconiosis takes the prediction precision of the MM-CNN analysis model as an objective function.
Preferably, the hybrid frog-leaping algorithm SFLA is adopted to optimize the hyper-parameters of the multi-classification MM-CNN pneumoconiosis analysis model, and the hyper-parameters comprise a 1D-CNN network layer number M, a 1D-CNN activation function, a 2D-CNN network layer number N, a 2D-CNN activation function, an optimizer and a learning rate, and the method specifically comprises the following steps:
initializing a frog population;
and (3) frog classification: sequencing the frogs in the population S according to the increasing sequence of the fitness, and recording the frog position P with the best fitness in the population SxIs F (1);
group division: partitioning the cultural genres according to the following formula;
Mk＝[Fk(j),fk(j)|Fk(j)＝F(k+m(j-1)),fk(j)＝f(k+m(j-1)),j＝1,2,…,n；k＝1,2,…,m]；
cultural gene inheritance evolution: each cultural genome Mk(k ═ 1,2, …, m) evolved independently from local search steps;
mixing culture gene bodies: after each cultural gene body is subjected to a round of local search, the population S is recombined, the population S is sorted in an increasing way according to fitness again, the optimal frogs in the population are updated, and the position P of the globally optimal frogs is recordedx；
And (3) checking a stopping condition: if the algorithm convergence condition is met, stopping the algorithm execution process; otherwise, returning to execute the group division.
Preferably, the local search process is implemented by the following specific steps:
step A: defining a calculator: setting mi to be 0, wherein mi is a counter of a culture gene body, and marking a serial number of a current evolutionary culture gene body; setting ni as 0, wherein ni is a counter of the independent evolution times, and marking and comparing whether the independent evolution times of the current culture gene body is smaller than the maximum independent evolution times or not;
and B: initializing a calculator: mi is mi + 1;
and C: initializing a calculator: ni + 1;
step D: according to pj2(n +1-j)/n (n +1), j 1,2, …, n to construct the subformulated gene:
step E: and (3) frog position updating: according to
Update the worst frog position and use F (q) ═ PW+ L to calculate a new location F (q), and if F (q) is in the feasible region, calculating a new fitness f (q); otherwise, entering the step F for execution; if the new fitness is better than the old fitness, i.e. a better result is generated, replacing the old F (q) with the new F (q), and proceeding to step H; otherwise, entering the step F for execution;
wherein L is the jumping step length of the frog with the worst fitness in the subformization gene body, and r is [0, 1 ]]Random number of intervals, PBFor the best location of the frog in the subformulation genome, PWThe worst frog position in the subformized genome, LmaxThe maximum jumping step length after the frog is infected;
step F: if step D can not produce better result, updating the worst frog position again according to
To calculate a jump step; if F (q) is in the feasible region, calculating new fitness f (q), otherwise, turning to the step G; if the new fitness is better than the old fitness, i.e. a better result is produced, the old F (q) is replaced by the new F (q), and step H is repeatedExecuting; otherwise, executing step G;
wherein, PXIs the global best position of the frog;
step G: randomly generating a new position of the frog: if the new position is not feasible and is not better than the old position, a new frog F (r) is randomly generated in the feasible region to replace the original frog, so as to stop the propagation of the defective cultural gene and calculate the fitness f (r);
step H: upgrading culture gene bodies: the worst frog in the sub-culture gene body is transferred and evolved to replace the worst frog in the culture gene body MmiAnd arranges M in descending order of fitnessmi；
Step I: checking the evolution times: if ni is less than n, skipping to the step C, and carrying out next inheritance evolution;
step J: checking culture gene number: if mi is less than m, skipping to the step B, and carrying out inheritance evolution of the next culture gene; otherwise, returning to the global search to mix cultural gene bodies.
Preferably, the basic information of the personal information includes basic information such as sex, age, height, weight, occupation history (including work units, departments (workshops), work types, harmful factors, protective measures and the like), tobacco and wine history, past history (including whether hypertension, diabetes, tuberculosis and the like exist, and time of the same) and the like of the person, and is used for assisting chest radiography analysis.
The occupational pneumoconiosis multi-modal analysis method based on deep learning provided by the invention has the following beneficial effects:
1. the individual difference of the operating personnel can be comprehensively considered by collecting the chest X-ray image information and the personal basic information of the personnel;
2. the adopted convolutional neural network algorithm has good classification performance, the information after considering the personal basic information word vector is one-dimensional data, the chest X-ray image information is two-dimensional data, corresponding one-dimensional convolutional networks and two-dimensional convolutional neural networks are respectively designed to extract the characteristics of multi-modal data, and an MM-CNN model suitable for analysis and diagnosis of occupational pneumoconiosis is constructed on the basis;
3. in the process of selecting the hyper-parameters of the MM-CNN model, a mixed frog-jumping algorithm is adopted to optimize the parameters of the 1D-CNN network layer number M, the 1D-CNN activation function, the 2D-CNN network layer number N, the 2D-CNN activation function, the optimizer and the learning rate of the MM-CNN occupational pneumoconiosis analysis and diagnosis model, so that the defect of manually selecting DF parameters is overcome;
4. the method can be used for rapidly detecting and analyzing the lung health condition and the occupational pneumoconiosis of special group personnel, so that accurate and real-time detection and analysis of the lung health of the personnel are realized, early warning of part of occupational pneumoconiosis is completed, and the life health of the personnel is guaranteed.
Drawings
In order to more clearly illustrate the embodiments of the present invention and the design thereof, the drawings required for the embodiments will be briefly described below. The drawings in the following description are only some embodiments of the invention and it will be clear to a person skilled in the art that other drawings can be derived from them without inventive effort.
FIG. 1 is a flowchart of a deep learning-based multimodal analysis method for occupational pneumoconiosis in embodiment 1 of the present invention;
FIG. 2 is a schematic diagram of a one-dimensional convolutional neural network structure;
FIG. 3 is a schematic diagram of a two-dimensional convolutional neural network structure;
FIG. 4 is a mixed frog leaping algorithm MM-CNN model hyper-parameter flow chart;
fig. 5 is a partial search flowchart.
Detailed Description
In order that those skilled in the art will better understand the technical solutions of the present invention and can practice the same, the present invention will be described in detail with reference to the accompanying drawings and specific examples. The following examples are only for illustrating the technical solutions of the present invention more clearly, and the protection scope of the present invention is not limited thereby.
Example 1
The invention provides a deep learning-based multi-modal analysis method for occupational pneumoconiosis, which is specifically shown in figure 1 and comprises the following steps: the method comprises the following steps:
s1, collecting chest X-ray image information and personal basic information of a person; the chest X-ray image acquisition system mainly comprises a chest X-ray image acquisition unit, a personal information acquisition unit and a chest X-ray image acquisition unit, wherein the chest X-ray image acquisition unit is mainly used for acquiring basic information of sex, age, height, weight, professional history (including work units, departments (workshops), work seeds, harmful factors, protective measures and the like), tobacco and wine history, past history (including whether hypertension, diabetes, tuberculosis and the like exist and time of the same) and the like of personnel by means of a direct Digital Radiography (DR) system;
during the acquisition process of the chest X-ray image information of a person, the following points need to be ensured:
firstly, the method comprises the following steps: requirements for camera position: when the chest is in a back-front standing position, the examinee should tightly attach the chest wall to the photographic frame, naturally separate the two feet, and rotate the two arms, so that the scapula does not overlap the lung field as much as possible;
secondly, the method comprises the following steps: requirements for source image distance: the source image distance is kept at 180 cm;
thirdly, the method comprises the following steps: requirements for focus: using a small focus;
fourthly: requirements for bulb position: adjusting the position of the bulb tube, wherein the center line is horizontal to the sixth thoracic vertebra;
fifth, the method comprises the following steps: requirements for exposure control: automatic exposure control is adopted, the exposure time is less than 100ms, and the exposure is carried out in a breath holding state after full air suction;
sixth: requirements for imaging voltage: the imaging voltage is controlled at 100-125 kV.
S2, carrying out word vectorization processing on the personal basic information;
specifically, in the word vectorization processing of the personal basic information, a skip-gram model in word2vec is adopted to perform word vectorization conversion of the personal basic information of the personnel, the size of a context window is set to be 10, the size of a word vector dimension is set to be 50, and the sampling size is set to be 1 e-3.
S3, respectively constructing a one-dimensional convolutional neural network 1D-CNN and a two-dimensional convolutional neural network 2D-CNN based on the personal information after word vectorization processing and the chest X-ray image information of the personnel, and establishing a multi-mode convolutional neural network MM-CNN model on the basis;
further, as shown in fig. 2, constructing a one-dimensional convolutional neural network 1D-CNN specifically includes: deepening the number of layers of the one-dimensional convolutional neural network is realized by connecting a plurality of one-dimensional convolutional units, the sizes of convolutional kernels are all set to be 3, when one convolutional layer is the L-th convolutional layer, the number of convolutional kernels is 8 multiplied by L, the one-dimensional pooling layer adopts a maximum pooling mode, and the pooling size is 2;
as shown in fig. 3, the two-dimensional convolutional neural network is constructed specifically as follows: the deepening of the number of layers of the two-dimensional convolutional neural network is realized by connecting a plurality of two-dimensional convolutional units, the sizes of convolutional kernels are all set to be 3 multiplied by 3, when one convolutional layer is the L-th convolutional layer, the number of the convolutional kernels is 8 multiplied by L, the two-dimensional pooling layer adopts a maximum pooling mode, and the pooling size is 2 multiplied by 2.
S4, taking the chest X-ray image information of the person and the personal information after word vectorization as the input of a multi-mode convolutional neural network MM-CNN model, establishing a multi-classification MM-CNN pneumoconiosis analysis model for occupational pneumoconiosis analysis, and forming an objective function under the condition of meeting corresponding constraints; specifically, the prediction accuracy of the MM-CNN analysis model is taken as an objective function.
S5, optimizing the hyper-parameters of the multi-classification MM-CNN pneumoconiosis analysis model by adopting a mixed frog-jumping algorithm SFLA;
in this embodiment, a mixed frog-leaping algorithm SFLA is used to optimize the hyper-parameters of the multi-classification MM-CNN pneumoconiosis analysis model, including the 1D-CNN network layer number M, the 1D-CNN activation function, the 2D-CNN network layer number N, the 2D-CNN activation function, the optimizer, and the learning rate, as shown in fig. 4, the specific steps are as follows:
s51: initializing a frog population;
s52: and (3) frog classification: sequencing the frogs in the population S according to the increasing sequence of the fitness, and recording the frog position P with the best fitness in the population SxIs F (1);
s53: group division: division of cultural genres according to the formula
Mk＝[Fk(j),fk(j)|Fk(j)＝F(k+m(j-1)),fk(j)＝f(k+m(j-1)),j＝1,2,…,n；k＝1,2,…,m]；
S54: cultural gene inheritance evolution: each cultural genome Mk(k-1, 2, …, m) evolved independently from the local search step.
Further, in this embodiment, as shown in fig. 5, the specific steps implemented in the local search process are as follows:
s541: defining a calculator: setting mi to be 0, wherein mi is a counter of the culture gene body, and marking the serial number of the current evolutionary culture gene body; and (5) setting ni as 0, wherein ni is a counter of the independent evolution times, and marking and comparing whether the independent evolution times of the current culture gene body is smaller than the maximum independent evolution times.
S542: initializing a calculator: mi is mi + 1;
s543: initializing a calculator: ni + 1;
s544: according to pj2(n +1-j)/n (n +1), j 1,2, …, n to construct the subformulated gene:
s545: and (3) frog position updating: according to
Update the worst frog position and use F (q) ═ PW+ L to calculate a new location F (q), and if F (q) is in the feasible region, calculating a new fitness f (q); otherwise, step S546 is executed. If the new fitness is better than the old fitness, i.e. a better result is generated, replacing the old f (q) with the new f (q), and proceeding to step S548; otherwise, the process proceeds to step S546.
Wherein L is the jumping step length of the frog with the worst fitness in the subformization gene body, and r is [0, 1 ]]Random number of intervals, PBFor the best location of the frog in the subformulation genome, PWThe worst frog position in the subformized genome, LmaxThe maximum jumping step length after the frog is infected;
s546: if step S544 fails to produce a good result, the worst frog position is updated again. According to
To calculate the jumping stepLong. If F (q) is in the feasible region, calculating new fitness f (q), otherwise, turning to the step S547; if the new fitness is better than the old fitness, i.e. a better result is produced, replacing the old f (q) with the new f (q), and performing step S548; otherwise, the step S547 is executed.
Wherein, PXIs the global best position of the frog;
s547: randomly generating a new position of the frog: if the new position is not feasible and is not better than the old position, a new frog F (r) is randomly generated in the feasible region to replace the original frog, so as to stop the propagation of the defective cultural gene and calculate the fitness f (r);
s548: upgrading culture gene bodies: the worst frog in the sub-culture gene body is transferred and evolved to replace the worst frog in the culture gene body MmiAnd arranges M in descending order of fitnessmi；
S549: checking the evolution times: if ni is less than n, jumping to step S543 to perform next inheritance evolution;
s5410: checking culture gene number: if mi is less than m, jumping to step S542, and carrying out inheritance evolution of the next culture gene; otherwise, returning to the global search to mix cultural gene bodies.
S55: mixing culture gene bodies: after each cultural gene body is subjected to a round of local search, the population S is recombined, the population S is sorted in an increasing way according to fitness again, the optimal frogs in the population are updated, and the position P of the globally optimal frogs is recordedx；
S56: and (3) checking a stopping condition: if the algorithm convergence condition is met, stopping the algorithm execution process; otherwise, return to step S53 is performed.
S6, analyzing chest X-ray image information and personal information after word vectorization processing of the personnel by adopting the optimized multi-classification MM-CNN pneumoconiosis analysis model, and outputting results of professional pneumoconiosis analysis.
The above-mentioned embodiments are only preferred embodiments of the present invention, and the scope of the present invention is not limited thereto, and any simple modifications or equivalent substitutions of the technical solutions that can be obviously obtained by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention.
Claims (7)
1. A multi-modal analysis method for occupational pneumoconiosis based on deep learning is characterized by comprising the following steps:
collecting chest X-ray image information and personal basic information of a person;
performing word vectorization processing on the personal basic information;
constructing a one-dimensional convolutional neural network 1D-CNN and a two-dimensional convolutional neural network 2D-CNN, and establishing a multi-mode convolutional neural network MM-CNN model on the basis;
taking chest X-ray image information of personnel and personal information after word vectorization as input of a multi-mode convolutional neural network MM-CNN model, establishing a multi-classification MM-CNN pneumoconiosis analysis model for occupational pneumoconiosis analysis, and forming an objective function under the condition of meeting corresponding constraints;
optimizing the hyper-parameters of the multi-classification MM-CNN pneumoconiosis analysis model by adopting a mixed frog-jumping algorithm SFLA;
and analyzing the chest X-ray image information and the personal information after word vectorization processing of the personnel by adopting the optimized multi-classification MM-CNN pneumoconiosis analysis model, and outputting the result of the occupational pneumoconiosis analysis.
2. The profound learning-based occupational pneumoconiosis multi-modal analysis method according to claim 1, wherein in the word vectorization processing of the personal basic information, a skip-gram model in word2vec is used for word vectorization conversion of the personal basic information, the size of a context window is set to 10, the size of a word vector dimension is set to 50, and the sampling size is set to 1 e-3.
3. The occupational pneumoconiosis multimodal analysis method based on deep learning of claim 1, wherein the constructing of the one-dimensional convolutional neural network 1D-CNN is specifically that the number of layers of the one-dimensional convolutional neural network is deepened by connecting a plurality of one-dimensional convolutional units, the sizes of convolutional kernels are all set to be 3, when one convolutional layer is the L-th convolutional layer, the number of convolutional kernels is 8 xL, the one-dimensional pooling layer adopts a maximum pooling mode, and the pooling size is 2;
the method for constructing the two-dimensional convolutional neural network specifically comprises the steps of deepening the layer number of the two-dimensional convolutional neural network by connecting a plurality of two-dimensional convolutional units, setting the sizes of convolutional kernels to be 3 multiplied by 3, and when one convolutional layer is the L-th convolutional layer, the number of the convolutional kernels is 8 multiplied by L, wherein the two-dimensional pooling layer adopts a maximum pooling mode, and the pooling size is 2 multiplied by 2.
4. The method of claim 1, wherein the MM-CNN model is established for multiple classifications of diagnosis and analysis of occupational pneumoconiosis, and the prediction accuracy of the MM-CNN model is used as an objective function.
5. The occupational pneumoconiosis multimodal analysis method based on deep learning of claim 1, wherein the mixed frogging SFLA (multiple classification of MM-CNN pneumoconiosis) algorithm is adopted to optimize the hyper-parameters of the multi-classification MM-CNN pneumoconiosis analysis model, and comprises a 1D-CNN network layer number M, a 1D-CNN activation function, a 2D-CNN network layer number N, a 2D-CNN activation function, an optimizer and a learning rate, and the method comprises the following specific steps:
initializing a frog population;
and (3) frog classification: sequencing the frogs in the population S according to the increasing sequence of the fitness, and recording the frog position P with the best fitness in the population SxIs F (1);
group division: partitioning the cultural genres according to the following formula;
Mk＝[Fk(j),fk(j)|Fk(j)＝F(k+m(j-1)),fk(j)＝f(k+m(j-1)),j＝1,2,…,n；k＝1,2,…,m]；
cultural gene inheritance evolution: each cultural genome Mk(k ═ 1,2, …, m) evolved independently from local search steps;
mixing culture gene bodies: in each cultural genomeAfter a round of local search, the population S is recombined, the population S is sorted in an increasing way according to the fitness again, the optimal frogs in the population are updated, and the position P of the globally optimal frogs is recordedx；
And (3) checking a stopping condition: if the algorithm convergence condition is met, stopping the algorithm execution process; otherwise, returning to execute the group division.
6. The deep learning-based multimodal analysis method for occupational pneumoconiosis according to claim 5, wherein the local search process is implemented by the following specific steps:
step A: defining a calculator: setting mi to be 0, wherein mi is a counter of a culture gene body, and marking a serial number of a current evolutionary culture gene body; setting ni as 0, wherein ni is a counter of the independent evolution times, and marking and comparing whether the independent evolution times of the current culture gene body is smaller than the maximum independent evolution times or not;
and B: initializing a calculator: mi is mi + 1;
and C: initializing a calculator: ni + 1;
step D: according to pj2(n +1-j)/n (n +1), j 1,2, …, n to construct the subformulated gene:
step E: and (3) frog position updating: according to
Update the worst frog position and use F (q) ═ PW+ L to calculate a new location F (q), and if F (q) is in the feasible region, calculating a new fitness f (q); otherwise, entering the step F for execution; if the new fitness is better than the old fitness, i.e. a better result is generated, replacing the old F (q) with the new F (q), and proceeding to step H; otherwise, entering the step F for execution;
wherein L is the jumping step length of the frog with the worst fitness in the subformization gene body, and r is [0, 1 ]]Random number of intervals, PBFor the best location of the frog in the subformulation genome, PWThe worst frog position in the subformized genome, LmaxThe maximum jumping step length after the frog is infected;
step F: if step D can not produce better result, updating the worst frog position again according to
To calculate a jump step; if F (q) is in the feasible region, calculating new fitness f (q), otherwise, turning to the step G; if the new fitness is better than the old fitness, i.e. a better result is produced, replacing the old F (q) with the new F (q), and executing step H; otherwise, executing step G;
wherein, PXIs the global best position of the frog;
step G: randomly generating a new position of the frog: if the new position is not feasible and is not better than the old position, a new frog F (r) is randomly generated in the feasible region to replace the original frog, so as to stop the propagation of the defective cultural gene and calculate the fitness f (r);
step H: upgrading culture gene bodies: the worst frog in the sub-culture gene body is transferred and evolved to replace the worst frog in the culture gene body MmiAnd arranges M in descending order of fitnessmi；
Step I: checking the evolution times: if ni is less than n, skipping to the step C, and carrying out next inheritance evolution;
step J: checking culture gene number: if mi is less than m, skipping to the step B, and carrying out inheritance evolution of the next culture gene; otherwise, returning to the global search to mix cultural gene bodies.
7. The deep learning-based multimodal analysis method for occupational pneumoconiosis according to claim 1, wherein the personal information basic information comprises sex, age, height, weight, occupational history, smoking and drinking history, and past history information of the person.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110579022.8A CN113255889B (en) | 2021-05-26 | 2021-05-26 | Multi-modal analysis method for occupational pneumoconiosis based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110579022.8A CN113255889B (en) | 2021-05-26 | 2021-05-26 | Multi-modal analysis method for occupational pneumoconiosis based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113255889A true CN113255889A (en) | 2021-08-13 |
| CN113255889B CN113255889B (en) | 2024-06-14 |
Family
ID=77184578
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110579022.8A Active CN113255889B (en) | 2021-05-26 | 2021-05-26 | Multi-modal analysis method for occupational pneumoconiosis based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113255889B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070118399A1 (en) * | 2005-11-22 | 2007-05-24 | Avinash Gopal B | System and method for integrated learning and understanding of healthcare informatics |
| CN106971198A (en) * | 2017-03-03 | 2017-07-21 | 北京市计算中心 | A kind of pneumoconiosis grade decision method and system based on deep learning |
| CN107679580A (en) * | 2017-10-21 | 2018-02-09 | 桂林电子科技大学 | A kind of isomery shift image feeling polarities analysis method based on the potential association of multi-modal depth |
| CA3000166A1 (en) * | 2017-04-03 | 2018-10-03 | Royal Bank Of Canada | Systems and methods for cyberbot network detection |
| CN109190699A (en) * | 2018-08-29 | 2019-01-11 | 西安电子科技大学 | A kind of more disease joint measurement methods based on multi-task learning |
| WO2019060450A1 (en) * | 2017-09-19 | 2019-03-28 | The Broad Institute, Inc. | Methods and systems for reconstruction of developmental landscapes by optimal transport analysis |
| CN109727637A (en) * | 2018-12-29 | 2019-05-07 | 陕西师范大学 | Method based on shuffled frog leaping algorithm identification key protein matter |
| CN110751562A (en) * | 2019-10-29 | 2020-02-04 | 山东科技大学 | Training optimization method for foreigner time sequence prediction |
| CN111368088A (en) * | 2020-03-31 | 2020-07-03 | 成都信息工程大学 | Text emotion classification method based on deep learning |
| CN111612096A (en) * | 2020-06-01 | 2020-09-01 | 南通大学 | Large-scale fundus image classification system training method based on Spark platform |
| CN111862136A (en) * | 2020-06-22 | 2020-10-30 | 南开大学 | Multi-modal nuclear magnetic image ischemic stroke lesion segmentation method based on convolutional neural network |
-
2021
- 2021-05-26 CN CN202110579022.8A patent/CN113255889B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070118399A1 (en) * | 2005-11-22 | 2007-05-24 | Avinash Gopal B | System and method for integrated learning and understanding of healthcare informatics |
| CN106971198A (en) * | 2017-03-03 | 2017-07-21 | 北京市计算中心 | A kind of pneumoconiosis grade decision method and system based on deep learning |
| CA3000166A1 (en) * | 2017-04-03 | 2018-10-03 | Royal Bank Of Canada | Systems and methods for cyberbot network detection |
| WO2019060450A1 (en) * | 2017-09-19 | 2019-03-28 | The Broad Institute, Inc. | Methods and systems for reconstruction of developmental landscapes by optimal transport analysis |
| CN107679580A (en) * | 2017-10-21 | 2018-02-09 | 桂林电子科技大学 | A kind of isomery shift image feeling polarities analysis method based on the potential association of multi-modal depth |
| CN109190699A (en) * | 2018-08-29 | 2019-01-11 | 西安电子科技大学 | A kind of more disease joint measurement methods based on multi-task learning |
| CN109727637A (en) * | 2018-12-29 | 2019-05-07 | 陕西师范大学 | Method based on shuffled frog leaping algorithm identification key protein matter |
| CN110751562A (en) * | 2019-10-29 | 2020-02-04 | 山东科技大学 | Training optimization method for foreigner time sequence prediction |
| CN111368088A (en) * | 2020-03-31 | 2020-07-03 | 成都信息工程大学 | Text emotion classification method based on deep learning |
| CN111612096A (en) * | 2020-06-01 | 2020-09-01 | 南通大学 | Large-scale fundus image classification system training method based on Spark platform |
| CN111862136A (en) * | 2020-06-22 | 2020-10-30 | 南开大学 | Multi-modal nuclear magnetic image ischemic stroke lesion segmentation method based on convolutional neural network |
Non-Patent Citations (5)
| Title |
|---|
| JIHAD S OBEID等: "Identifying and Predicting Intentional Self-Harm in Electronic Health Record Clinical Notes: Deep Learning Approach", 《JMIR MED INFORM》, vol. 8, no. 7, 30 July 2020 (2020-07-30), pages 17784 * |
| KAI BIAN等: "CEEMD: A New Method to Identify Mine Water Inrush Based on the Signal Processing and Laser-Induced Fluorescence", 《IEEE ACCESS》, vol. 8, 5 June 2020 (2020-06-05), pages 107076 - 107086, XP011793791, DOI: 10.1109/ACCESS.2020.3000333 * |
| 卞凯: "基于体征参数的数据归约矿工健康状况辨识方法研究", 《中国博士学位论文全文数据库 医药卫生科技辑》, no. 02, 15 February 2023 (2023-02-15), pages 055 - 17 * |
| 李益兵等: "基于SFLA改进卷积神经网络的滚动轴承故障诊断", 《振动与冲击》, vol. 39, no. 24, 31 December 2020 (2020-12-31), pages 187 - 193 * |
| 邓琛: "基于语谱图和卷积神经网络的语音情绪识别研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》, no. 02, 15 February 2022 (2022-02-15), pages 136 - 153 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113255889B (en) | 2024-06-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Basu et al. | Deep learning for screening covid-19 using chest x-ray images | |
| Militante et al. | Pneumonia detection through adaptive deep learning models of convolutional neural networks | |
| El Gannour et al. | COVID-19 Detection on x-ray images using a combining mechanism of pre-trained CNNs | |
| Adams et al. | Prediction of outcome in Parkinson’s disease patients from DAT SPECT images using a convolutional neural network | |
| Çallı et al. | FRODO: Free rejection of out-of-distribution samples: application to chest x-ray analysis | |
| CN117174330A (en) | IgA nephropathy patient treatment scheme recommendation method based on machine learning | |
| CN111145902A (en) | Asthma diagnosis method based on improved artificial neural network | |
| CN113255889A (en) | Occupational pneumoconiosis multi-modal analysis method based on deep learning | |
| Lyakhov et al. | Neural network system for analyzing statistical factors of patients for predicting the survival of dental implants | |
| Dawood | A new method based CNN combined with genetic algorithm and support vector machine for COVID-19 detection by analyzing X-ray images | |
| Patel | Classification of COVID-19 from chest X-ray images using a deep convolutional neural network | |
| Raje et al. | Automated Diagnosis of Pneumonia through Capsule Network in conjunction with ResNet50v2 model | |
| Jabbar et al. | Deep learning based classification of wrist cracks from X-ray imaging | |
| Parvez et al. | Applications in the Field of Bioinformatics | |
| CN111466877B (en) | LSTM network-based oxygen reduction state prediction method | |
| Glotov et al. | Pulmonary fibrosis progression prognosis using machine learning | |
| Kogilavani et al. | AI powered COVID-19 detection system using non-contact sensing technology and deep learning techniques | |
| Mellal et al. | Android App based on CNN for Covid-19 Detection using Chest X-ray Images | |
| Ramyea et al. | Predictive analysis of pulmonary fibrosis progression using gradient boosting classifier and DICOM images | |
| Vong et al. | Pneumothorax segmentation in chest X-rays using UNet++ and efficientnet | |
| Vetrithangam et al. | prediction of pneumonia disease from x-ray images using a modified resnet152v2 deep learning model | |
| Kaboudan et al. | AI-Driven Medical Imaging Platform: Advancements in Image Analysis and Healthcare Diagnosis | |
| Varalakshmi et al. | Diminishing fall-out and miss-rate in the classification of lung diseases using deep learning techniques | |
| Lydia et al. | Unveiling Insights: Harnessing AI for Lung Disease Detection and Classification | |
| Wajgi et al. | Optimized tuberculosis classification system for chest X‐ray images: Fusing hyperparameter tuning with transfer learning approaches |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |