CN113255889B - Multi-modal analysis method for occupational pneumoconiosis based on deep learning - Google Patents
Multi-modal analysis method for occupational pneumoconiosis based on deep learning Download PDFInfo
- Publication number
- CN113255889B CN113255889B CN202110579022.8A CN202110579022A CN113255889B CN 113255889 B CN113255889 B CN 113255889B CN 202110579022 A CN202110579022 A CN 202110579022A CN 113255889 B CN113255889 B CN 113255889B
- Authority
- CN
- China
- Prior art keywords
- frog
- cnn
- pneumoconiosis
- genome
- cultural
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 206010035653 pneumoconiosis Diseases 0.000 title claims abstract description 62
- 238000004458 analytical method Methods 0.000 title claims abstract description 54
- 238000013135 deep learning Methods 0.000 title claims abstract description 15
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 97
- 238000000034 method Methods 0.000 claims abstract description 17
- 238000011976 chest X-ray Methods 0.000 claims abstract description 15
- 230000006870 function Effects 0.000 claims abstract description 15
- 238000012545 processing Methods 0.000 claims abstract description 11
- 230000009191 jumping Effects 0.000 claims abstract description 10
- 238000011176 pooling Methods 0.000 claims description 18
- 230000008569 process Effects 0.000 claims description 12
- 230000004913 activation Effects 0.000 claims description 8
- 238000003745 diagnosis Methods 0.000 claims description 8
- 238000010276 construction Methods 0.000 claims description 5
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims description 3
- 241000208125 Nicotiana Species 0.000 claims description 2
- 235000002637 Nicotiana tabacum Nutrition 0.000 claims description 2
- 230000002950 deficient Effects 0.000 claims description 2
- 108090000623 proteins and genes Proteins 0.000 claims description 2
- 210000004072 lung Anatomy 0.000 abstract description 9
- 230000036541 health Effects 0.000 abstract description 7
- 238000010223 real-time analysis Methods 0.000 abstract description 2
- 238000011897 real-time detection Methods 0.000 abstract description 2
- 210000000038 chest Anatomy 0.000 description 9
- 238000002601 radiography Methods 0.000 description 9
- 239000000428 dust Substances 0.000 description 6
- 239000003245 coal Substances 0.000 description 5
- 201000010099 disease Diseases 0.000 description 4
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 4
- 238000013473 artificial intelligence Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 239000011435 rock Substances 0.000 description 3
- 206010008479 Chest Pain Diseases 0.000 description 2
- VYPSYNLAJGMNEJ-UHFFFAOYSA-N Silicium dioxide Chemical compound O=[Si]=O VYPSYNLAJGMNEJ-UHFFFAOYSA-N 0.000 description 2
- 210000001132 alveolar macrophage Anatomy 0.000 description 2
- 238000005422 blasting Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 206010011224 Cough Diseases 0.000 description 1
- 208000000059 Dyspnea Diseases 0.000 description 1
- 206010013975 Dyspnoeas Diseases 0.000 description 1
- 206010020772 Hypertension Diseases 0.000 description 1
- 208000028571 Occupational disease Diseases 0.000 description 1
- 208000002193 Pain Diseases 0.000 description 1
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 210000003123 bronchiole Anatomy 0.000 description 1
- 239000004566 building material Substances 0.000 description 1
- 239000002817 coal dust Substances 0.000 description 1
- 206010012601 diabetes mellitus Diseases 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 230000003862 health status Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 210000002540 macrophage Anatomy 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000001681 protective effect Effects 0.000 description 1
- 208000005069 pulmonary fibrosis Diseases 0.000 description 1
- 230000003014 reinforcing effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 210000001991 scapula Anatomy 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 208000013220 shortness of breath Diseases 0.000 description 1
- 239000000377 silicon dioxide Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 208000024891 symptom Diseases 0.000 description 1
- 210000000115 thoracic cavity Anatomy 0.000 description 1
- 210000000779 thoracic wall Anatomy 0.000 description 1
- 210000001519 tissue Anatomy 0.000 description 1
- 201000008827 tuberculosis Diseases 0.000 description 1
- 230000005641 tunneling Effects 0.000 description 1
- 238000005406 washing Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10116—X-ray image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30061—Lung
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a multi-modal analysis method of occupational pneumoconiosis based on deep learning, which belongs to the field of pneumoconiosis analysis and comprises the following steps: acquiring chest X-ray image information and personal basic information of a person; performing word vectorization processing on the personal basic information; constructing a one-dimensional convolutional neural network and a two-dimensional convolutional neural network, and establishing a multimode convolutional neural network MM-CNN model on the basis; taking the two information as input of a multi-mode convolutional neural network MM-CNN model, establishing a multi-classification MM-CNN pneumoconiosis analysis model, and forming an objective function under the condition of meeting corresponding constraint; optimizing hyper-parameters of a multi-classification MM-CNN pneumoconiosis analysis model by adopting a mixed jumping frog algorithm SFLA; and analyzing chest X-ray image information of the personnel and personal information subjected to word vectorization by adopting an optimized multi-classification MM-CNN pneumoconiosis analysis model, and outputting an analysis result. The method can realize accurate and real-time detection and analysis of the lung health of the personnel and complete early warning of part of occupational pneumoconiosis.
Description
Technical Field
The invention belongs to the field of pneumoconiosis analysis, and particularly relates to a multi-modal analysis method for professional pneumoconiosis based on deep learning.
Background
In recent years, the safety production in China is continuously and stably improved, the death number of the production safety accident is continuously and rapidly reduced for many years, but the situation facing the occupational health work is still very serious. Occupational pneumoconiosis is taken as one of occupational diseases and is mainly distributed in industrial industries such as coal, colored, mechanical, building materials, light industry and the like. For example, in the coal mine production process, many links such as rock roadway blasting, rock roadway loading, rock roadway tunneling, coal roadway blasting, coal roadway reinforcing, coal separation and transportation can generate many dust (mainly including coal dust and silicon dust), and the suction of excessive dust is the root cause of pneumoconiosis. Pneumoconiosis is a disease in which fibrosis of lung tissue occurs as a result of a worker inhaling a large amount of free silica and other dust for a long period of time during a job. Most of the dust is discharged, but a part remains in bronchioles and alveoli for a long time, and is continuously phagocytized by alveolar macrophages, and the dust-swallowing macrophages are main causative factors. A series of studies have shown that after pneumoconiosis has formed, residual dust in the lungs continues to act on alveolar macrophages, which is a major cause of continued development of lesions in pneumoconiosis patients despite the release of dust. The common symptoms of pneumoconiosis patients are chest distress, chest pain, shortness of breath, cough, general weakness, loss of work ability even for heavy people, and finally lung failure, kneeling and death, which is not seen.
Pneumoconiosis is an incurable disease, no specific medicine capable of curing pneumoconiosis exists in the world at present, the pain of a patient can only be relieved to a certain extent by washing the lung, the development of the disease is slowed down, and the disease cannot be fundamentally reversed. And the early detection of the lung health condition of the personnel can realize the early recognition of the occupational pneumoconiosis, improve the early discovery rate of the occupational pneumoconiosis, and reduce the serious burden brought to the personnel by the occupational pneumoconiosis. Therefore, it is necessary and important to develop a diagnostic device and an analysis method for professional pneumoconiosis of this particular group, which can realize accurate and real-time measurement and analysis of the lung health status of the personnel, and has important significance for early warning of part of professional pneumoconiosis and ensuring of the life health of the personnel.
At present, judging occupational pneumoconiosis mainly depends on diagnosis of occupational pneumoconiosis (GBZ-2015), and doctors perform diagnosis analysis of pneumoconiosis by comparing and analyzing chest films of patients with X-ray diagnosis standard films of pneumoconiosis on the basis of relevant diagnosis standards and principles and depending on personal experience. With the continuous development and application of artificial intelligence algorithms, the application of the artificial intelligence algorithms to the auxiliary analysis of chest radiography image data becomes a research hotspot, and a series of successful applications are achieved. However, the chest radiography is affected by various factors such as the position of an illuminated object, exposure conditions, operation errors, films and the like in the chest radiography acquisition process, so that the image quality of the chest radiography is uneven in level, the difficulty in feature extraction and analysis is high, and the defects of low identification precision, long time consumption and the like exist in the process of directly using a traditional artificial intelligence algorithm for chest radiography analysis. With the proposal of deep learning theory and the improvement of numerical computing equipment, convolutional neural networks (Convolutional Neural Network, CNN) are rapidly developed, and CNN is one of the best modes of feature extraction in image processing and text analysis and is applied to the fields of computer vision, natural language processing and the like. Although the CNN model has excellent performance, how to design a suitable network structure for specific application problems and find optimal model parameters is a difficult problem in the CNN application process.
In view of the above, the invention provides a multi-modal analysis method of occupational pneumoconiosis based on deep learning.
Disclosure of Invention
In order to overcome the defects in the prior art, the invention provides a multi-modal analysis method for professional pneumoconiosis based on deep learning.
In order to achieve the above object, the present invention provides the following technical solutions:
a multi-modal analysis method of occupational pneumoconiosis based on deep learning, comprising the steps of:
Acquiring chest X-ray image information and personal basic information of a person;
performing word vectorization processing on the personal basic information;
Constructing a One-dimensional convolutional neural network (One-dimensional Convolutional Neural Network, 1D-CNN) and a Two-dimensional convolutional neural network (Two-dimensional Convolutional Neural Network, 2D-CNN), and constructing a Multi-mode convolutional neural network (Multi-modal Convolution Neural Network, MM-CNN) model on the basis;
Taking chest X-ray image information of personnel and personal information after word vectorization processing as input of a multi-mode convolutional neural network MM-CNN model, establishing a multi-classification MM-CNN pneumoconiosis analysis model for occupational pneumoconiosis analysis, and forming an objective function under the condition of meeting corresponding constraint;
Optimizing hyper-parameters of a multi-classification MM-CNN pneumoconiosis analysis model by adopting a mixed jumping frog algorithm (Shuffled Frog LeapingAlgorithm, SFLA);
and analyzing chest X-ray image information of personnel and personal information subjected to word vectorization by adopting an optimized multi-classification MM-CNN pneumoconiosis analysis model, and outputting a result of occupational pneumoconiosis analysis.
Preferably, in the word vectorization processing of the personal basic information, a skip-gram model in word2vec is adopted to perform word vectorization conversion of the personal basic information of the person, the context window size is set to 10, the word vector dimension size is set to 50, and the sampling size is set to 1e-3.
Preferably, the construction of the one-dimensional convolutional neural network 1D-CNN is specifically to realize deepening of the layer number of the one-dimensional convolutional neural network by connecting a plurality of one-dimensional convolutional units, wherein the sizes of convolutional kernels are all set to 3, when one convolutional layer is an L-th convolutional layer, the number of the convolutional kernels is 8×L, and a one-dimensional pooling layer adopts a maximum pooling mode, and the pooling size is 2;
The construction of the two-dimensional convolutional neural network is specifically that the deepening of the layer number of the two-dimensional convolutional neural network is realized by connecting a plurality of two-dimensional convolutional units, the size of convolution kernels is set to be 3 multiplied by 3, when one convolution layer is an L-th convolution layer, the number of the convolution kernels is 8 multiplied by L, the two-dimensional pooling layer adopts a maximum pooling mode, and the pooling size is 2 multiplied by 2.
Preferably, the method establishes a multi-classification MM-CNN model for occupational pneumoconiosis analysis and diagnosis, and takes the prediction precision of the MM-CNN analysis model as an objective function.
Preferably, the super parameters of the multi-classification MM-CNN pneumoconiosis analysis model optimized by adopting the mixed leapfrog algorithm SFLA include 1D-CNN network layer number M, 1D-CNN activation function, 2D-CNN network layer number N, 2D-CNN activation function, optimizer and learning rate, and the specific steps are as follows:
initializing frog population;
Frog classification: sorting the frog in the population S according to the order of increasing fitness, and recording the position P x of the frog with the best fitness in the population S as F (1);
Group division: dividing cultural genome according to the following formula;
Mk=[Fk(j),fk(j)|Fk(j)=F(k+m(j-1)),fk(j)=f(k+m(j-1)),j=1,2,…,n;k=1,2,…,m];
Cultural genome inheritance evolution: each cultural genome M k (k=1, 2, …, M) evolved independently according to the local search step;
Mixing the culture genome: after each cultural genome is subjected to one round of local search, the population S is recombined, the optimal frog in the population is updated again according to the incremental sequencing of the fitness, and the position P x of the global optimal frog is recorded;
check stop condition: if the algorithm convergence condition is met, stopping the algorithm execution process; otherwise, the group division is performed back.
Preferably, the local search process is implemented by the following specific steps:
Step A: definition calculator: let mi=0, where mi is the counter of the cultural genome, marking the sequence number of the current evolving cultural genome; setting ni=0, wherein ni is a counter of independent evolution times, marking and comparing whether the independent evolution times of the current cultural genome is less than the maximum independent evolution times;
and (B) step (B): initializing a calculator: mi=mi+1;
Step C: initializing a calculator: ni=ni+1;
Step D: constructing a subculture genome from p j = 2 (n+1-j)/n (n+1), j = 1,2, …, n:
Step E: frog position update: according to Updating the worst frog position, calculating a new position F (q) by using F (q) =P W +L, and calculating a new fitness F (q) if F (q) is in a feasible region; otherwise, entering the step F for execution; if the new fitness is better than the old fitness, that is, a better result is generated, replacing the old F (q) with the new F (q), and transferring to the step H for execution; otherwise, entering the step F for execution;
Wherein L is the jump step length of the frog with the worst adaptability in the sub-culture genome, r is a random number in the interval of [0,1], P B is the best position of the frog in the sub-culture genome, P W is the worst position of the frog in the sub-culture genome, and L max is the maximum jump step length after the frog is infected;
step F: if step D does not produce a good result, the worst frog position is updated again according to Calculating a jump step length; if F (q) is in a feasible domain, calculating new fitness F (q), otherwise, turning to the step G; if the new fitness is better than the old fitness, i.e. a better result is produced, replacing the old F (q) with the new F (q), and converting to step H for execution; otherwise, step G is carried out;
wherein P X is the global best position of the frog;
step G: randomly generating new positions of the frog: if the new position is not feasible and is not better than the old position, randomly generating a new frog F (r) in the feasible region to replace the original frog so as to terminate the propagation of the defective cultural genes and calculate the fitness F (r);
Step H: upgrading cultural genome: after inheritance evolution, the worst frog in the culture genome replaces the source of the frog in the culture genome M mi, and M mi is arranged in descending order of fitness;
step I: checking the evolution times: if ni is less than n, jumping to the step C, and carrying out the next inheritance evolution;
Step J: checking the cultural genome number: if mi is less than m, jumping to the step B, and carrying out the inheritance evolution of the next cultural genome; otherwise, returning to the global search to mix cultural genome.
Preferably, the personal information base information includes the sex, age, height, weight, occupation history (including work unit, department (workshop), work species, harmful factors, protective measures, etc.), tobacco and wine history, past history (including whether there is hypertension, diabetes, tuberculosis, etc. and time) etc. of the personnel, and is used for assisting chest radiography analysis.
The multi-modal analysis method of professional pneumoconiosis based on deep learning provided by the invention has the following beneficial effects:
1. the individual difference of the operators can be comprehensively considered by collecting chest X-ray image information and personal basic information of the operators;
2. The adopted convolutional neural network algorithm has good classification performance, and the information after the personal basic information word vector is considered to be one-dimensional data, the chest X-ray image information is two-dimensional data, and the corresponding one-dimensional convolutional network and two-dimensional convolutional neural network are respectively designed to extract the characteristics of multi-modal data, so that an MM-CNN model suitable for professional pneumoconiosis analysis and diagnosis is constructed on the basis;
3. in the process of carrying out the hyper-parameter selection of the MM-CNN model, a mixed jumping frog algorithm is adopted to carry out parameter optimization of the 1D-CNN network layer number M, the 1D-CNN activation function, the 2D-CNN network layer number N, the 2D-CNN activation function, the optimizer and the learning rate of the MM-CNN occupational pneumoconiosis analysis and diagnosis model, so that the defect of manually selecting DF parameters is solved;
4. The method can be used for carrying out rapid detection and analysis on the lung health condition and occupational pneumoconiosis of special group personnel, realizing accurate and real-time detection and analysis on the lung health of the personnel, completing early warning of partial occupational pneumoconiosis and guaranteeing the life health of the personnel.
Drawings
In order to more clearly illustrate the embodiments of the present invention and the design thereof, the drawings required for the embodiments will be briefly described below. The drawings in the following description are only some of the embodiments of the present invention and other drawings may be made by those skilled in the art without the exercise of inventive faculty.
FIG. 1 is a flow chart of a multi-modal analysis method of professional pneumoconiosis based on deep learning according to embodiment 1 of the invention;
FIG. 2 is a schematic diagram of a one-dimensional convolutional neural network;
FIG. 3 is a schematic diagram of a two-dimensional convolutional neural network structure;
FIG. 4 is a flow chart of the hybrid frog-leaping algorithm optimization MM-CNN model super parameter;
Fig. 5 is a partial search flow chart.
Detailed Description
The present invention will be described in detail below with reference to the drawings and the embodiments, so that those skilled in the art can better understand the technical scheme of the present invention and can implement the same. The following examples are only for more clearly illustrating the technical aspects of the present invention, and are not intended to limit the scope of the present invention.
Example 1
The invention provides a multi-modal analysis method of occupational pneumoconiosis based on deep learning, which is specifically shown in figure 1 and comprises the following steps: the method comprises the following steps:
S1, acquiring chest X-ray image information and personal basic information of a person; the chest radiography acquisition unit is mainly used for acquiring chest radiography images of personnel by means of a direct digital radiography (Digital Radiography, DR) system, and the personal information acquisition unit is mainly used for acquiring basic information such as sex, age, height, weight, occupation history (including work units, department areas (workshops), work types, harmful factors, protective measures and the like), tobacco history and wine history, existing history (including whether hypertension, diabetes, tuberculosis and the like exist or not, time of the same) and the like of the personnel;
In the process of acquiring chest X-ray image information of personnel, the following points need to be ensured:
First: requirements for the photographic position: the chest is positioned at the front back, the chest wall of the testee should be closely attached to the photographing frame, the feet are naturally separated, and the arms are rotated inwards, so that the scapula is prevented from overlapping the lung field as much as possible;
Second,: requirements for source image distance: the source image distance is kept to be 180cm;
Third,: requirements for focus: using a small focal point;
fourth,: requirements for bulb position: adjusting the position of the bulb tube, wherein the central line is at the level of the sixth thoracic vertebra;
Fifth,: requirements for exposure control: the automatic exposure control is adopted, the exposure time is less than 100ms, and the exposure is carried out in a fully-aspirated breath-hold state;
Sixth: requirements for image pickup voltage: the photographing voltage is controlled to be 100-125kV.
S2, carrying out word vectorization processing on the personal basic information;
Specifically, in the word vectorization processing of the personal basic information, a skip-gram model in word2vec is adopted to perform word vectorization conversion of the personal basic information of the personnel, the size of a context window is set to 10, the size of a word vector dimension is set to 50, and the sampling size is set to 1e-3.
S3, respectively constructing a one-dimensional convolutional neural network 1D-CNN and a two-dimensional convolutional neural network 2D-CNN based on the personal information subjected to word vectorization processing and chest X-ray image information of the personnel, and establishing a multi-mode convolutional neural network MM-CNN model on the basis;
Further, a one-dimensional convolutional neural network 1D-CNN is constructed as shown in FIG. 2, specifically: the deepening of the layer number of the one-dimensional convolutional neural network is realized by connecting a plurality of one-dimensional convolutional units, the size of the convolutional kernels is set to be 3, when one convolutional layer is an L-th convolutional layer, the number of the convolutional kernels is 8 multiplied by L, the one-dimensional pooling layer adopts a maximum pooling mode, and the pooling size is 2;
the construction of a two-dimensional convolutional neural network is shown in fig. 3, and specifically comprises the following steps: the deepening of the layer number of the two-dimensional convolutional neural network is realized by connecting a plurality of two-dimensional convolutional units, the sizes of the convolutional kernels are all set to be 3 multiplied by 3, when one convolutional layer is an L-th convolutional layer, the number of the convolutional kernels is 8 multiplied by L, the two-dimensional pooling layer adopts a maximum pooling mode, and the pooling size is 2 multiplied by 2.
S4, taking chest X-ray image information of a person and personal information after word vectorization as input of a multi-mode convolutional neural network MM-CNN model, establishing a multi-classification MM-CNN pneumoconiosis analysis model for occupational pneumoconiosis analysis, and forming an objective function under the condition of meeting corresponding constraint; specifically, the prediction accuracy of the MM-CNN analysis model is used as an objective function.
S5, optimizing super parameters of the multi-classification MM-CNN pneumoconiosis analysis model by adopting a mixed jumping frog algorithm SFLA;
In this embodiment, the super parameters of the multi-classification MM-CNN pneumoconiosis analysis model are optimized by using the mixed leapfrog algorithm SFLA, including 1D-CNN network layer number M, 1D-CNN activation function, 2D-CNN network layer number N, 2D-CNN activation function, optimizer and learning rate, as shown in fig. 4, the specific steps are as follows:
S51: initializing frog population;
S52: frog classification: sorting the frog in the population S according to the order of increasing fitness, and recording the position P x of the frog with the best fitness in the population S as F (1);
s53: group division: dividing cultural genome according to the following
Mk=[Fk(j),fk(j)|Fk(j)=F(k+m(j-1)),fk(j)=f(k+m(j-1)),j=1,2,…,n;k=1,2,…,m];
S54: cultural genome inheritance evolution: each cultural genome M k (k=1, 2, …, M) evolved independently according to the local search procedure.
Further, in this embodiment, as shown in fig. 5, the specific steps of the local search process are as follows:
S541: definition calculator: let mi=0, where mi is the counter of the cultural genome; let ni=0, where ni is a counter of independent evolutions, flag and compare if the independent evolutions of the current cultural genome are less than the maximum independent evolutions.
S542: initializing a calculator: mi=mi+1;
s543: initializing a calculator: ni=ni+1;
s544: constructing a subculture genome from p j = 2 (n+1-j)/n (n+1), j = 1,2, …, n:
s545: frog position update: according to Updating the worst frog position, calculating a new position F (q) by using F (q) =P W +L, and calculating a new fitness F (q) if F (q) is in a feasible region; otherwise, step S546 is performed. If the new fitness is better than the old fitness, i.e. a better result is produced, replacing the old F (q) with the new F (q), and proceeding to step S548 for execution; otherwise, the flow advances to step S546 to execute.
Wherein L is the jump step length of the frog with the worst adaptability in the sub-culture genome, r is a random number in the interval of [0,1], P B is the best position of the frog in the sub-culture genome, P W is the worst position of the frog in the sub-culture genome, and L max is the maximum jump step length after the frog is infected;
s546: if step S544 fails to produce a good result, the worst frog position is updated again. According to To calculate the jump step size. If F (q) is in the feasible region, calculating a new fitness F (q), otherwise turning to step S547; if the new fitness is better than the old fitness, i.e. a better result is produced, replacing the old F (q) with the new F (q), and turning to step S548 for execution; otherwise, step S547 is performed.
Wherein P X is the global best position of the frog;
S547: randomly generating new positions of the frog: if the new position is not feasible and is not better than the old position, randomly generating a new frog F (r) in the feasible region to replace the original frog so as to terminate the propagation of the defective cultural genes and calculate the fitness F (r);
s548: upgrading cultural genome: after inheritance evolution, the worst frog in the culture genome replaces the source of the frog in the culture genome M mi, and M mi is arranged in descending order of fitness;
s549: checking the evolution times: if ni is less than n, jumping to step S543 to carry out next inheritance evolution;
s5410: checking the cultural genome number: if mi is less than m, jumping to step S542, and carrying out the next cultural genome inheritance evolution; otherwise, returning to the global search to mix cultural genome.
S55: mixing the culture genome: after each cultural genome is subjected to one round of local search, the population S is recombined, the optimal frog in the population is updated again according to the incremental sequencing of the fitness, and the position P x of the global optimal frog is recorded;
S56: check stop condition: if the algorithm convergence condition is met, stopping the algorithm execution process; otherwise, the process returns to step S53.
S6, analyzing chest X-ray image information and personal information after word vectorization treatment by adopting an optimized multi-classification MM-CNN pneumoconiosis analysis model, and outputting a result of occupational pneumoconiosis analysis.
The above embodiments are merely preferred embodiments of the present invention, the protection scope of the present invention is not limited thereto, and any simple changes or equivalent substitutions of technical solutions that can be obviously obtained by those skilled in the art within the technical scope of the present invention disclosed in the present invention belong to the protection scope of the present invention.
Claims (6)
1. A multi-modal analysis method for occupational pneumoconiosis based on deep learning, which is characterized by comprising the following steps:
Acquiring chest X-ray image information and personal basic information of a person;
performing word vectorization processing on the personal basic information;
Constructing a one-dimensional convolutional neural network 1D-CNN and a two-dimensional convolutional neural network 2D-CNN, and establishing a multi-mode convolutional neural network MM-CNN model on the basis;
Taking chest X-ray image information of personnel and personal information after word vectorization processing as input of a multi-mode convolutional neural network MM-CNN model, establishing a multi-classification MM-CNN pneumoconiosis analysis model for occupational pneumoconiosis analysis, and forming an objective function under the condition of meeting corresponding constraint;
optimizing hyper-parameters of a multi-classification MM-CNN pneumoconiosis analysis model by adopting a mixed jumping frog algorithm SFLA;
Adopting an optimized multi-classification MM-CNN pneumoconiosis analysis model to analyze chest X-ray image information of personnel and personal information after word vectorization treatment, and outputting a professional pneumoconiosis analysis result;
the super-parameters of the multi-classification MM-CNN pneumoconiosis analysis model are optimized by adopting a mixed jumping frog algorithm SFLA, and the super-parameters comprise 1D-CNN network layer number M, 1D-CNN activation function, 2D-CNN network layer number N, 2D-CNN activation function, optimizer and learning rate, and the specific steps are as follows:
initializing frog population;
Frog classification: sorting the frog in the population S according to the order of increasing fitness, and recording the position P x of the frog with the best fitness in the population S as F (1);
Group division: dividing cultural genome according to the following formula;
Mk=[Fk(j),fk(j)|Fk(j)=F(k+m(j-1)),fk(j)=f(k+m(j-1)),j=1,2,…,n;k=1,2,…,m]
Cultural genome inheritance evolution: each cultural genome M k (k=1, 2, …, M) evolved independently according to the local search step;
Mixing the culture genome: after each cultural genome is subjected to one round of local search, the population S is recombined, the optimal frog in the population is updated again according to the incremental sequencing of the fitness, and the position P x of the global optimal frog is recorded;
check stop condition: if the algorithm convergence condition is met, stopping the algorithm execution process; otherwise, the group division is performed back.
2. The deep learning based professional pneumoconiosis multimodal analysis method as claimed in claim 1, wherein in the word vectorization processing of the personal basic information, a skip-gram model in word2vec is used to perform word vectorization conversion of personal basic information of a person, the context window size is set to 10, the word vector dimension size is set to 50, and the sampling size is set to 1e-3.
3. The multi-modal analysis method of occupational pneumoconiosis based on deep learning as claimed in claim 1, wherein the construction of one-dimensional convolutional neural network 1D-CNN is specifically implemented by connecting a plurality of one-dimensional convolutional units to deepen the layer number of the one-dimensional convolutional neural network, the size of the convolutional kernels is set to 3, when one convolutional layer is the L-th convolutional layer, the number of the convolutional kernels is 8×l, the one-dimensional pooling layer adopts a maximum pooling mode, and the pooling size is 2;
The construction of the two-dimensional convolutional neural network is specifically that the deepening of the layer number of the two-dimensional convolutional neural network is realized by connecting a plurality of two-dimensional convolutional units, the size of convolution kernels is set to be 3 multiplied by 3, when one convolution layer is an L-th convolution layer, the number of the convolution kernels is 8 multiplied by L, the two-dimensional pooling layer adopts a maximum pooling mode, and the pooling size is 2 multiplied by 2.
4. The deep learning based multi-modal analysis method of occupational pneumoconiosis according to claim 1, wherein the multi-classification MM-CNN model for occupational pneumoconiosis analysis and diagnosis is established, and the prediction accuracy of the MM-CNN analysis model is used as an objective function.
5. The depth learning based professional pneumoconiosis multimodal analysis method as claimed in claim 1, wherein the local search process is implemented as the following steps:
Step A: definition calculator: let mi=0, where mi is the counter of the cultural genome, marking the sequence number of the current evolving cultural genome; setting ni=0, wherein ni is a counter of independent evolution times, marking and comparing whether the independent evolution times of the current cultural genome is less than the maximum independent evolution times;
and (B) step (B): initializing a calculator: mi=mi+1;
Step C: initializing a calculator: ni=ni+1;
Step D: constructing a subculture genome from p j = 2 (n+1-j)/n (n+1), j = 1,2, …, n:
Step E: frog position update: according to Updating the worst frog position, calculating a new position F (q) by using F (q) =P W +L, and calculating a new fitness F (q) if F (q) is in a feasible region; otherwise, entering the step F for execution; if the new fitness is better than the old fitness, that is, a better result is generated, replacing the old F (q) with the new F (q), and transferring to the step H for execution; otherwise, entering the step F for execution;
Wherein L is the jump step length of the frog with the worst adaptability in the sub-culture genome, r is a random number in the interval of [0,1], P B is the best position of the frog in the sub-culture genome, P W is the worst position of the frog in the sub-culture genome, and L max is the maximum jump step length after the frog is infected;
step F: if step D does not produce a good result, the worst frog position is updated again according to Calculating a jump step length; if F (q) is in the feasible region, calculating new fitness F (q), otherwise, turning to step G; if the new fitness is better than the old fitness, i.e. a better result is produced, replacing the old F (q) with the new F (q), and converting to step H for execution; otherwise, entering the step G for execution;
wherein P X is the global best position of the frog;
step G: randomly generating new positions of the frog: if the new position is not feasible and is not better than the old position, randomly generating a new frog F (r) in the feasible region to replace the original frog so as to terminate the propagation of the defective cultural genes and calculate the fitness F (r);
Step H: upgrading cultural genome: after inheritance evolution, the worst frog in the culture genome replaces the source of the frog in the culture genome M mi, and M mi is arranged in descending order of fitness;
step I: checking the evolution times: if ni is less than n, jumping to the step C, and carrying out the next inheritance evolution;
Step J: checking the cultural genome number: if mi is less than m, jumping to the step B, and carrying out the inheritance evolution of the next cultural genome; otherwise, returning to the global search to mix cultural genome.
6. The deep learning based professional pneumoconiosis multimodal analysis method according to claim 1, wherein the personal basic information comprises sex, age, height, weight, professional history, tobacco and wine history and past history information of the person.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110579022.8A CN113255889B (en) | 2021-05-26 | 2021-05-26 | Multi-modal analysis method for occupational pneumoconiosis based on deep learning |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110579022.8A CN113255889B (en) | 2021-05-26 | 2021-05-26 | Multi-modal analysis method for occupational pneumoconiosis based on deep learning |
Publications (2)
Publication Number | Publication Date |
---|---|
CN113255889A CN113255889A (en) | 2021-08-13 |
CN113255889B true CN113255889B (en) | 2024-06-14 |
Family
ID=77184578
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202110579022.8A Active CN113255889B (en) | 2021-05-26 | 2021-05-26 | Multi-modal analysis method for occupational pneumoconiosis based on deep learning |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN113255889B (en) |
Family Cites Families (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070118399A1 (en) * | 2005-11-22 | 2007-05-24 | Avinash Gopal B | System and method for integrated learning and understanding of healthcare informatics |
CN106971198A (en) * | 2017-03-03 | 2017-07-21 | 北京市计算中心 | A kind of pneumoconiosis grade decision method and system based on deep learning |
CA3000166A1 (en) * | 2017-04-03 | 2018-10-03 | Royal Bank Of Canada | Systems and methods for cyberbot network detection |
US20200224172A1 (en) * | 2017-09-19 | 2020-07-16 | The Broad Institute, Inc. | Methods and systems for reconstruction of developmental landscapes by optimal transport analysis |
CN107679580B (en) * | 2017-10-21 | 2020-12-01 | 桂林电子科技大学 | Heterogeneous migration image emotion polarity analysis method based on multi-mode depth potential correlation |
CN109190699B (en) * | 2018-08-29 | 2022-12-30 | 西安电子科技大学 | Multi-disease joint determination method based on multi-task learning |
CN109727637B (en) * | 2018-12-29 | 2023-09-05 | 陕西师范大学 | Method for identifying key proteins based on mixed frog-leaping algorithm |
CN110751562A (en) * | 2019-10-29 | 2020-02-04 | 山东科技大学 | Training optimization method for foreigner time sequence prediction |
CN111368088A (en) * | 2020-03-31 | 2020-07-03 | 成都信息工程大学 | Text emotion classification method based on deep learning |
CN111612096A (en) * | 2020-06-01 | 2020-09-01 | 南通大学 | Large-scale fundus image classification system training method based on Spark platform |
CN111862136A (en) * | 2020-06-22 | 2020-10-30 | 南开大学 | Multi-modal nuclear magnetic image ischemic stroke lesion segmentation method based on convolutional neural network |
-
2021
- 2021-05-26 CN CN202110579022.8A patent/CN113255889B/en active Active
Non-Patent Citations (2)
Title |
---|
基于体征参数的数据归约矿工健康状况辨识方法研究;卞凯;《中国博士学位论文全文数据库 医药卫生科技辑》;20230215(第02期);E055-17 * |
基于语谱图和卷积神经网络的语音情绪识别研究;邓琛;《中国优秀硕士学位论文全文数据库 信息科技辑》;20220215(第02期);I136-153 * |
Also Published As
Publication number | Publication date |
---|---|
CN113255889A (en) | 2021-08-13 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Wang et al. | Skeletal maturity recognition using a fully automated system with convolutional neural networks | |
CN110738654B (en) | Key point extraction and bone age prediction method in hip joint image | |
Abdollahi et al. | An artificial intelligence system for detecting the types of the epidemic from X-rays: artificial intelligence system for detecting the types of the epidemic from X-rays | |
El Gannour et al. | COVID-19 Detection on x-ray images using a combining mechanism of pre-trained CNNs | |
CN110755085A (en) | Motion function evaluation method and equipment based on joint mobility and motion coordination | |
Çallı et al. | FRODO: Free rejection of out-of-distribution samples: application to chest x-ray analysis | |
CN111145902A (en) | Asthma diagnosis method based on improved artificial neural network | |
CN116705300A (en) | Medical decision assistance method, system and storage medium based on sign data analysis | |
CN113255889B (en) | Multi-modal analysis method for occupational pneumoconiosis based on deep learning | |
Naskinova | Transfer learning with NASNet-Mobile for Pneumonia X-ray classification | |
CN117838097A (en) | Spinal deformity analysis system based on key point detection | |
Patel | Classification of COVID-19 from chest X-ray images using a deep convolutional neural network | |
CN117237304A (en) | Bone age assessment method and system for children with autonomous learning ability | |
CN112259232A (en) | VTE risk automatic evaluation system based on deep learning | |
CN116543455A (en) | Method, equipment and medium for establishing parkinsonism gait damage assessment model and using same | |
CN110755084B (en) | Motion function assessment method and device based on active-passive and staged actions | |
CN111466877B (en) | LSTM network-based oxygen reduction state prediction method | |
Bhat | Automated detection of COVID-19 from X-ray images using deep convolutional neural networks | |
Dhanapala et al. | Transfer learning techniques with SVM for covid-19 disease prediction based on chest X-ray images | |
Kaboudan et al. | AI-Driven Medical Imaging Platform: Advancements in Image Analysis and Healthcare Diagnosis | |
Mellal et al. | Android App based on CNN for Covid-19 Detection using Chest X-ray Images | |
TWI678709B (en) | Disease prediction method through a big database formed by data mining of neural network | |
Ramyea et al. | Predictive analysis of pulmonary fibrosis progression using gradient boosting classifier and DICOM images | |
Mahajan et al. | Feasibility of Cervical Spine Fracture Detection Using Non-Conventional Slices | |
Gehlot et al. | C-LVQ: A Convolutional Neural Network with Learning Vector Quantization for the Diagnosis of Covid-19 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |