CN110334843B - Time-varying attention improved Bi-LSTM hospitalization and hospitalization behavior prediction method and device - Google Patents
Time-varying attention improved Bi-LSTM hospitalization and hospitalization behavior prediction method and device Download PDFInfo
- Publication number
- CN110334843B CN110334843B CN201910324147.9A CN201910324147A CN110334843B CN 110334843 B CN110334843 B CN 110334843B CN 201910324147 A CN201910324147 A CN 201910324147A CN 110334843 B CN110334843 B CN 110334843B
- Authority
- CN
- China
- Prior art keywords
- hospitalization
- prediction
- data
- lstm
- hospitalizing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
- G16H40/20—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the management or administration of healthcare resources or facilities, e.g. managing hospital staff or surgery rooms
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/70—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Business, Economics & Management (AREA)
- Public Health (AREA)
- General Business, Economics & Management (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- Biomedical Technology (AREA)
- Medical Informatics (AREA)
- Primary Health Care (AREA)
- Economics (AREA)
- Epidemiology (AREA)
- General Physics & Mathematics (AREA)
- Strategic Management (AREA)
- Human Resources & Organizations (AREA)
- Operations Research (AREA)
- Biophysics (AREA)
- Databases & Information Systems (AREA)
- Quality & Reliability (AREA)
- Pathology (AREA)
- Marketing (AREA)
- Entrepreneurship & Innovation (AREA)
- Game Theory and Decision Science (AREA)
- Development Economics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Tourism & Hospitality (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Medical Treatment And Welfare Office Work (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The present disclosure provides time-varying attention-improving Bi-LSTM hospitalization behavior prediction methods and apparatus. The hospitalization and hospitalization behavior prediction method comprises the following steps: extracting the characteristic that the correlation degree with the hospitalizing and hospitalizing behaviors is greater than a preset correlation threshold value from the mass medical insurance data; constructing Bi-LSTM by using the extracted hospitalization and hospitalization behavior characteristics and the corresponding weight thereof; updating the weight value of each hospitalization state prediction data in the Bi-LSTM based on the hospitalization state prediction data acquired by the Bi-LSTM and hospital-disease attraction data pre-generated by an attention mechanism; constructing a time adjustment function; outputting a multi-period and multi-state hospitalizing and hospitalizing state prediction vector; constructing a softmax prediction function by using the hospitalization and hospitalization state prediction vector; calculating a loss function of the output value of the softmax prediction function, and completing the training of the model by adopting a learning parameter of backward propagation training Bi-LSTM; and after the model training is finished, outputting the prediction result of the experimental sample set, comparing the prediction result with the actual hospitalizing behavior, and feeding back and updating the weight value of the hospitalizing state prediction data.
Description
Technical Field
The disclosure belongs to the field of medical insurance information processing, and particularly relates to a time-varying attention-improved Bi-LSTM hospitalization and hospitalization behavior prediction method and device.
Background
The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.
Hospitalization refers to the process of seeking medical assistance to alleviate symptoms or cure a disease when an individual develops symptoms. Due to the difference between the medical technical level and the medical insurance reimbursement policy of each hospital, the individual tends to visit hospitals with high medical level and relatively preferential medical insurance reimbursement, so that the problems of high medical service load, unbalanced medical resource distribution, serious medical insurance fund payment and the like of some hospitals are caused, and the medical resource waste is further caused by the long-term mode. Therefore, it is urgently needed to mine potential medical rules based on massive medical insurance data, further construct a medical behavior prediction model, and provide decision support for medical insurance fund allocation and medical resource optimization scheduling.
In recent years, with the advance of the technical field of medical insurance information, the conventional prediction method regards each medical record as one feature and then constructs a prediction model using the feature as an input. Because the Recurrent Neural Networks (RNNs) can realize time sequence correlation prediction based on the hospitalization record sequence, they are widely used in hospitalization prediction modeling, such as hospitalization diagnosis prediction tasks, risk prediction tasks, and the like. However, the RNNs cannot effectively solve the long-time dependent expression relationship between the hospitalization sequences, and the prediction performance of the RNNs model is reduced when the time interval of the hospitalization sequences of the patient is too large. In addition, the inventor finds that due to the special regularity of hospitalizing behaviors, such as different attractions of hospitalizing trends generated by specialties of hospitals, few hospitalizing records in a long time or a plurality of hospitalizing records in a short time, the influence of special attention and time effectiveness of hospitals on the hospitalizing behaviors is ignored in the traditional hospitalizing behavior prediction method, and therefore the prediction effectiveness of the hospitalizing behaviors is difficult to realize.
Disclosure of Invention
In a first aspect of the disclosure, a time-varying attention improved Bi-LSTM hospitalization and hospitalization behavior prediction method is provided, which is based on a hidden layer of a bidirectional Long-Short Term Memory (Bi-LSTM) model to store influence states of hospitalization records at different times on subsequent prediction, so that prediction of hospitalization and hospitalization states is realized, then an attention mechanism is introduced to realize attraction weight adjustment of hospital-diseases, and in addition, hospitalization state weight optimization based on pathological rules is completed in combination with a time adjustment factor, so that prediction of future hospitalization and hospitalization behaviors is realized in combination with a time function and the attention mechanism improved Bi-LSTM model.
The technical scheme of the time-varying attention improved Bi-LSTM hospitalization and hospitalization behavior prediction method in the first aspect of the disclosure is as follows:
a time-varying attention-improving Bi-LSTM hospitalization behavior prediction method comprising:
extracting the characteristic that the correlation degree with the hospitalizing and hospitalizing behaviors is greater than a preset correlation threshold value from the mass medical insurance data;
constructing Bi-LSTM by using the extracted hospitalization behavior characteristics and the corresponding weights thereof to realize the hospitalization state prediction of the hospitalization information in the positive and negative directions;
receiving hospital hospitalizing data transmitted by a bottom layer based on hospitalizing state prediction data acquired by the Bi-LSTM, generating a hospital-disease matching weight by adopting an attention mechanism, and updating the weight value of each hospitalizing state prediction data in the Bi-LSTM by adopting the attention mechanism;
forming a pathological time rule based on bottom layer transmission data, constructing a time adjusting function according to the change of the influence degree of hospitalization behavior characteristics along with the increase of the time distance, and multiplying the time adjusting function by the weight value of each hospitalization state prediction data, thereby updating the weight value of each stage of hospitalization state prediction data in the Bi-LSTM and outputting a multi-period multi-state hospitalization state prediction vector;
constructing a softmax prediction function by using the hospitalization and hospitalization state prediction vector; calculating a loss function of the output value of the softmax prediction function, and training the learning parameters of the Bi-LSTM by adopting a back propagation algorithm to complete the training of the model;
after the model training is finished, the prediction result of the experimental sample set is output, the comparison with the actual hospitalization behavior is carried out, and the bottom layer data information is fed back and updated, so that the attention mechanism and the data weight value of the time adjustment function are continuously optimized, and the hospitalization behavior prediction is continuously perfected.
A second aspect of the disclosure provides a time-varying attention-improving Bi-LSTM hospitalization visit behavior prediction apparatus.
A time-varying attention-improving Bi-LSTM hospitalization and hospitalization behavior prediction apparatus comprising:
the characteristic extraction module is used for extracting characteristics of which the correlation degree with hospitalization and hospitalization behaviors is greater than a preset correlation threshold value from the mass medical insurance data;
the Bi-LSTM construction module is used for constructing the Bi-LSTM by utilizing the extracted hospitalization behavior characteristics and the corresponding weights thereof so as to realize the hospitalization state prediction of the hospitalization information in the positive and negative directions;
the weight updating module is used for receiving hospital hospitalizing data transmitted by a bottom layer on the one hand based on the hospitalizing state prediction data acquired by the Bi-LSTM on the other hand, generating hospital-disease matching weights by adopting an attention mechanism and updating the weight values of the hospitalizing state prediction data in the Bi-LSTM by adopting the attention mechanism;
the time adjustment function construction module is used for constructing a time adjustment function according to a pathological time rule generated by the bottom data statistical mean value and the change of the influence degree of hospitalization and hospitalization behavior characteristics along with the increase of the time distance;
the state prediction output module is used for multiplying the time adjustment function by the weight value of each hospitalization state prediction data respectively so as to output a multi-period and multi-state hospitalization state prediction vector;
the prediction model training module is used for constructing a softmax prediction function by utilizing the hospitalization state prediction vector; calculating a loss function of the output value of the softmax prediction function, and training the learning parameters of the Bi-LSTM by adopting a back propagation algorithm to complete the training of the model;
and the feedback updating module is used for outputting the prediction result of the experimental sample set after the model training is finished, comparing the prediction result with the actual hospitalizing behavior, feeding back and updating the bottom layer data information, and continuously optimizing the weight value of the hospitalizing state prediction data of the attention mechanism and the time adjusting function so as to continuously perfect the hospitalizing behavior prediction.
A third aspect of the present disclosure provides a computer-readable storage medium.
A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the in-patient and medical-care behavior prediction method described above.
A fourth aspect of the disclosure provides a computer device.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the hospitalization and hospitalization behavior prediction method when executing the program.
The beneficial effects of this disclosure are:
(1) according to the method, the hospitalization and hospitalization behavior characteristics are extracted by adopting a Relief algorithm through analysis of the hospitalization and hospitalization behavior data in recent years, and the weight values of all the characteristics are updated based on the relevant statistical vectors of the screening characteristics, so that the characteristics with high correlation degree with the hospitalization and hospitalization behavior are extracted on one hand, and the accuracy of the hospitalization and hospitalization behavior prediction is improved based on weight distribution on the other hand;
(2) aiming at the characteristics that hospitalizing records have time sequence and long-term dependence, the selection intention of future hospitalizing behaviors of a patient is predicted by adopting Bi-LSTM, and on one hand, the input of the previous time point is stored in a memory mode in an internal circulation mode and is used as input data of next prediction output; on the other hand, a 'gate' operation concept is adopted, output errors generated in the chain transmission process flow in the network in a parameter mode, the problem of gradient explosion or gradient disappearance is avoided, and therefore long-term dependence correlation stability of hospitalization records is kept;
(3) the method aims to avoid the loss of the effectiveness of low-level data at the bottom layer when high-level abstract prediction data is extracted by a hidden layer in the prediction process of Bi-LSTM, so that the historical hospitalizing behaviors are quantized by adopting an attention mechanism and a pathological time rule, the prediction result of the hospitalizing behaviors at future hospitalizing is improved by hospital-disease attraction and time distance, on one hand, the attraction of different hospitals to different patient diseases is obviously enhanced by adopting the attention mechanism, on the other hand, the pathological regularity (such as short-term effect of viral influenza, cancer recurrence period and the like) is enhanced by a time adjusting function, and the updated degree of hospitalizing behavior states at different moments is determined by the attention score and the time function together, so that the prediction performance of the model is improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the disclosure, illustrate embodiments of the disclosure and together with the description serve to explain the disclosure and are not to limit the disclosure.
Fig. 1 is an overall flowchart of a hospitalization and hospitalization behavior prediction method provided by an embodiment of the present disclosure;
fig. 2 is a flowchart of characteristic extraction of hospitalization and hospitalization behaviors based on a Relief algorithm provided by an embodiment of the present disclosure;
FIG. 3 is a diagram of a processing process of a data stream with a Bi-LSML core according to an embodiment of the present disclosure;
fig. 4 is a flow chart of an embodiment of hospitalization travel behavior prediction provided by an embodiment of the present disclosure;
FIG. 5 is a Bi-LSTM training hidden state parameter diagram provided by an embodiment of the present disclosure;
fig. 6 is a comparison graph of hidden state weighted average accuracy based on pneumonia hospitalization sequences provided by the embodiments of the present disclosure.
Fig. 7 is a schematic structural diagram of a hospitalization and hospitalization behavior prediction apparatus according to an embodiment of the present disclosure.
Detailed Description
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the disclosure. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present disclosure. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
As shown in fig. 1 and fig. 4, the method for predicting hospitalization and hospitalization behavior of the time-varying attention-improving Bi-LSTM of the present embodiment includes:
A. the method comprises the steps of collecting medical insurance-related mass data including historical data and current hospitalizing data from information of the insured person and hospital information, and carrying out data preprocessing including data cleaning, missing data completion, data definition and data storage on the obtained mass medical insurance data.
Specifically, medical history data is obtained as an application example based on a social security system of a certain region in a certain city, the 2012-2016 historical medical data of the region is used as a training sample, the 2017-year historical data is used as an experimental sample, medical categories are divided into 21 categories according to an ICD-10 standard, and method verification is respectively performed from two aspects of medical seeking inside the region and medical seeking outside the region, as shown in table 1, a statistical example is provided for 4 disease data sets with the total cost of the historical medical data being earlier.
TABLE 1 basic statistics of hospitalization category data sets
Collecting 2012-2016 insured personnel samples in a certain region of a certain city, and researching factors influencing hospitalization and hospitalization behaviors from two aspects of insured people and hospitals respectively to obtain the following main factors: age, sex, insurance category, economic income, disease category, hospital level, treatment complexity of hospital treatment (average daily cost, average number of hospitalizations), treatment rate, maximum number of hospitalizations of hospital in a period of time, and part of factor sets that may influence hospitalization behavior in acquiring massive medical insurance data are shown in table 2:
TABLE 2 partial set of factors in the medical insurance data that may affect hospitalization
B. The mass medical insurance data are subjected to normalized processing, feature selection influencing hospitalizing factors is carried out based on filter selection (Relevant Features, Relief), redundancy of Features is eliminated, and Features with high correlation degree with hospitalizing behaviors are extracted.
The data preprocessing is carried out on the acquired data, the missing value processing and the data normalization are included, the chi-square test is adopted for carrying out simple correlation analysis, the randomness of repeated influence factors and factor selection is eliminated, the complexity of the problem is reduced, the redundancy of the characteristics is eliminated, and more information variables are selected to improve the accuracy and the efficiency of the prediction model.
For the acquired medical insurance data sample, based on filtering selection, feature selection influencing hospitalizing factors is carried out, redundancy of features is eliminated, and features with high correlation degree with hospitalization hospitalizing behaviors are extracted, during hospitalization and hospitalization, relevant statistics between the filtering features and the hospitalization behaviors are changed along with changes of time, for example, the treatment level (average daily average cost, average number of hospitalization days) of hospital treatment diseases, hospital grade and economic income of patients have larger influence on the hospitalization behaviors of the patients, and relevant statistics between 2016 year part of screening features and hospitalization behaviors are calculated by a Relief algorithm, and are shown in Table 3.
TABLE 3.2016 screening characteristics and related statistics for partial hospitalization
Specifically, as shown in fig. 2, a general generation process of performing feature selection affecting hospitalizing factors based on filtering selection (release Features, release) in step B is as follows:
B1. defining all hospitalizing class codes in medical insurance data as c1,c2,…,cc,…,c|C|e.C, where | C | is the number of hospitalization category encodings, assuming there are N patient sample data with T for the Nth patient sample data(n)A medical record, then a patient sample data can be represented by the medical sequence asEvery medical record packageContaining a set of feature vectors x ∈ R|C|;
B2. Standardizing the hospitalizing and hospitalizing behavior sample data, and carrying out normalization processing on original medical insurance data X by adopting a min-max standardization method, wherein the value of characteristic data is [0,1 ];
B3. given a sample set of hospitalization categories { (X)1,cc),(X2,cc),...,(XI,cc) For each sample XiI e (1, 2.. I) contains J (1,2, … J …, J) feature attributes, XiFirstly, searching the nearest neighbor sample X in the same kind of samplesi,nhCounting m as its guess neighbor, and locking X 'from other sample'i,nhAs Xi,nmNamely, guess neighbors, and similarly, guess neighbors and guess neighbors of k samples are respectively found. Then, its correlation statistic δ is calculated for the feature jjThe calculation formula is as follows:
in the formula (I), the compound is shown in the specification,represents a sample XiThe value vector on the characteristic attribute j, the same wayAnd depending on the type of the feature attribute j, if the attribute j is discrete, thenTime of flightOtherwise, the value is 1; if the feature attribute j is continuous, then
B4. For the feature attribute j, if XiWith its guessing neighbor Xi,nhThe smaller the distance is, the closer X is guessed by the distancei,nmThe larger the distance of (d), the larger its associated statistical vector δjThe larger the feature attribute j, the stronger the ability to distinguish the medical categories, and δjPerforming descending order arrangement, setting a threshold tau, and taking the feature of which the correlation statistic is larger than the threshold tau as a screening feature;
B5. relevant statistical vector delta based on screening characteristicsjUpdating the weight vector W of all the characteristics, and defining the formula of the weight W of the characteristic j as follows:
in the formula, p (c)c) Indicates that the random sample belongs to ccProportion of class, ccIn order to be in the category of medical treatment,
and outputting the screened features and the feature weight vector W corresponding to the screened features.
C. Extracting according to the screened hospitalization behavior characteristics, extracting a characteristic vector set of a historical hospitalization sample by adopting embedding mapping, constructing the Bi-LSTM to realize the hospitalization state prediction of the hospitalization information in the positive and negative directions, further setting a hospitalization state prediction threshold, and outputting a hospitalization state prediction result, as shown in FIG. 3.
According to the selected characteristic extraction of hospitalization behavior, extracting a characteristic vector set of a historical hospitalization sample by adopting embedding mapping, training Bi-LSTM to acquire hospitalization state prediction data of hospitalization information in forward and backward directions, and adjusting hidden state parameters of the model.
Specifically, an Adadelta optimizer is adopted in the training model, the random distribution proportion of a sample data set is 0.75,0.1 and 0.15, the sample data set is respectively used as a training set, a verification set and an experiment set, the initial theta and u values are respectively 1 and 0.1, the dimension m of an Embedding layer is 100, the dimension of the hospitalization and hospitalization state of the LSTM based on the disease category grade is initially set to be 105, and the threshold value of the iteration times is 200 times.
The general generation process of constructing the Bi-LSTM to realize the prediction of the hospitalization and hospitalization state in the positive and negative directions in the step C is as follows:
C1. based on the sample characteristics obtained in the step B, sample characteristic extraction is carried out on the historical training samples, the imbedding layer is adopted to map the characteristic data into index representation, a characteristic-index mapping matrix is constructed, and if a hospitalizing sequence X is giveni(i ═ 1,2, …, | C |), and obtaining a corresponding low-dimensional dense vector set v ∈ R by embedding mappingKAs follows:
v=ATx (14)
in the formula, K is the dimension of the embedding layer, and A belongs to R|C|·KRepresenting a feature-index mapping matrix;
C2. based on a low-dimensional feature vector set v, constructing an LSTM consisting of an input gate, a candidate memory unit, an output gate and a forgetting gate, wherein the model construction principle process is as follows:
ft=sigmoid(Wf·[ht-1,vt]+bf) (15)
it=sigmoid(Wi·[ht-1,vt]+bi) (16)
Ct=tanh(Wc·[ht-1,vt]+bc) (17)
Ct=ft×Ct-1+it×Ct(18)
ot=sigmoid(Wo·[ht-1,vt]+bo) (19)
ht=ot×tanh(Ct) (20)
in the formula, vtInput vector representing time t,ht-1Output vector representing last time instant, { W }f,Wi,Wc,WoIs the weight coefficient matrix of the corresponding cell, { bf,bi,bc,boThe displacement vector of the corresponding unit is used as the displacement vector, sigmoid is an activation function, the formula (15) is a forgetting gate data processing process, and the forgetting gate value at the time t is defined by vtAnd ht-1Calculating and obtaining, and reserving or abandoning the data record; equation (16) activates W with sigmoid functioni·[ht-1,vt]+biThe value of the cell state, equation (17) is the value to obtain the candidate memory cell; formula (18) is ft、itFor candidate memory cell Ct-1、CtCarrying out value adjustment; the expressions (19) and (20) are defined by h at time tt-1And vtInternal circulation and updating to obtain output h of LSTM's last hospitalization visit statust;
Forward LSTM reads from v based on constructed LSTM1To vTCalculating a series of forward hospitalization and hospitalization prediction states by the hospitalization sequence
C3. Reverse LSTM reading from v based on the LSTM constructed in step C2TTo v1Calculating a series of reverse hospitalization prediction states by the hospitalization sequence
C4. Setting hospitalization state parameter threshold based on Bi-LSTM outputted hospitalization state data, and obtaining final hospitalization state vector H by adopting data weighted summationt,
In the formula (I), the compound is shown in the specification,andthe predicted data of the hospitalization and hospitalization status respectively representing the forward direction and the backward direction at the time t, because only the forward direction data can be obtained in the actual hospitalization and hospitalization status prediction process, the forward direction predicted data is taken as a prediction quantitative standard, the backward direction data is taken as a threshold value setting standard, the values are shown in figure 5,for the final hospitalization and hospitalization status vector, H, obtained from the forward and reverse directionstTo the Attention mechanism.
D. Based on the hospitalization status prediction data acquired by Bi-LSTM, combining with a hospital-disease weight matrix omega acquired by probability statistics of hospitalization distribution data of each hospital transmitted from a bottom layer, updating the weight value of each hospital-disease prediction data by adopting an Attention (Attention) mechanism, namely:
ut=tanh(ωHt+b) (1)
in the formula, ω ∈ RL×|C|And b ∈ RLRespectively representing a weight matrix and a basic matrix, L is the number of hospitals, C is the disease category, utRepresenting the hospital-disease attention vector, T representing the number of hospital-disease attention generated at time T, atRepresents the weight of Hospital-disease normalized by equation (2), δ being atTo HtThe adjustment result of the weight;
E. considering that hospitalization and hospitalization behaviors change along with the lengthening of the distance, a time adjustment factor Δ t is introduced to construct a time adjustment function, which is obtained by formula (4):
Δt=τi-τj(5)
where σ denotes a sigmoid function.
In the formula (I), the compound is shown in the specification,andwhich represents the parameters of the learning process,for medical record cnThe initial degree of influence of (a) is,showing the influence degree changing along with time, obtaining the time-varying rule of various diseases according to the statistics of the underlying medical insurance data, and delta t is the hospitalizing record taujAnd the next medical record τ to be predictediTime interval between, and using sigmoid function, i.e. sigma, willΔ t transitions between 0-1;
combining with hospital-disease matching weight obtained by the Attention mechanism, the final output vector of the hospitalization and hospitalization stateComprises the following steps:
specifically, on the basis of hospitalization and hospitalization state prediction data acquired by Bi-LSTM, an Attention (Attention) mechanism is adopted to update the weighted value of each hospitalization and hospitalization state prediction data, a time adjustment factor delta t is introduced to construct a time adjustment function in consideration of the fact that the influence degree of hospitalization and hospitalization behavior characteristics changes along with the time distance, and the hospital-disease state weight acquired by the Attention mechanism is combined, so that the final output vector of the hospitalization behavior state is obtainedTaking the longest hospitalization record pneumonia data set-30 as an example, firstly selecting the latest hospitalization sequences for hospitalization and hospitalization behavior prediction, and then sequentially increasing the hospitalization sequences according to the hospitalization time sequence for prediction respectively, as shown in fig. 6, with the increase of the number of hospitalization records, the prediction performance of the Bi-LSTM model with time-varying attention improvement constructed in the patent is increased, but the traditional Bi-LSTM model is not combined with the attention influence and the time factor, so that the prediction performance is reduced.
F. Constructing a softmax prediction function based on a final output vector H of a hidden state output by historical sample training set data, and enabling a historical sample x to beiGenerated hidden state vector HiThe probability of classification into category j is:
in the formula, Wc∈R2LAnd bc∈RLObtaining learning parameters of a softmax regression prediction function based on historical training sample set data in a mode of iteratively minimizing a target function, and turning to the step G;
G. and (3) adopting cross entropy as a loss function, and if y is a real class distribution, defining the loss function as follows:
in combination with the weight distribution rule of the Attention mechanism, the objective function is,
wherein T is the length of the hospitalization sequence, α is a hyper-parameter determining the relative importance of these intermediate targets, obtained by the Attention mechanism;
continuously minimizing loss (y', y) through iterative solution of the step F and the step G, thereby obtaining learning parameters of the Bi-LSTM with time-varying attention improvement;
H. carrying out hospitalization and hospitalization behavior prediction on a sample to be predicted and outputting an Attention mechanismInputting a softmax layer for result prediction, wherein the prediction result is as follows:
pushing the prediction result, and comparing the result with the actual hospitalizing behavior;
carrying out hospitalization and hospitalization behavior prediction on a sample to be predicted and outputting an Attention mechanismInputting a softmax layer for result prediction, pushing a prediction result, comparing the result with an actual hospitalization and hospitalization behavior, and comparing the accuracy of the hospitalization and hospitalization behavior prediction based on the hospitalization behavior of 4 diseases in Table 4:
TABLE 4 accuracy of hospitalization and hospitalization behavior prediction
Based on the results in table 4, the prediction accuracy of the method proposed in this embodiment is higher than that of other prediction methods when the physician visits the inside or outside of the region.
I. Based on the comparison between the prediction result and the actual result, the extenstion mechanism is adjusted by interpretable feedback of the prediction result, the hospitalizing rule obtained by the statistics of the bottom-layer personal hospitalizing data is updated and perfected, the weight of each dimension in the hospital-disease matching is sorted according to the reverse order, and then K data with the top weight are selected, as shown in the following:
argsort(at[;.i])[1:K](11)
in the formula, at[;.i]The attention scores of the respective dimensions in the ith medical record are represented,by analyzing K hospitalizing records with the front weights, the importance degree of the hospitalizing records on hospitalizing behavior prediction can be fed back and adjusted, so that historical sample data is continuously updated, and the prediction accuracy of the method is improved.
As shown in fig. 7, the present embodiment provides a time-varying attention-improving Bi-LSTM hospitalization and hospitalization behavior prediction apparatus, comprising:
(1) the characteristic extraction module is used for extracting characteristics of which the correlation degree with hospitalization and hospitalization behaviors is greater than a preset correlation threshold value from the mass medical insurance data;
specifically, the feature extraction module is further configured to:
extracting medical insurance data samples similar to the same type and different types from the medical insurance data subjected to the standardized processing based on the characteristics, and calculating characteristic-related statistics;
and setting a related statistic threshold value as a preset relevance threshold value, and screening out features related to hospitalization and hospitalization behaviors.
(2) The Bi-LSTM construction module is used for constructing the Bi-LSTM by utilizing the extracted hospitalization behavior characteristics and the corresponding weights thereof so as to realize the hospitalization state prediction of the hospitalization information in the positive and negative directions;
(3) the weight updating module is used for updating the weight value of the hospitalization state prediction data in the Bi-LSTM by adopting an attention mechanism based on the hospitalization state prediction data acquired by the Bi-LSTM and combining the hospitalization statistical data of each hospital transmitted by the bottom layer;
(4) the time adjustment function building module is used for forming a pathological time rule based on bottom layer transmission data and building a time adjustment function according to the change of the influence degree of hospitalization and hospitalization behavior characteristics along with the increase of time distance;
(5) the state prediction output module is used for respectively multiplying the updated weight values of the hospitalization and hospitalization state prediction data by using a time adjustment function so as to update the weight values of the hospitalization and hospitalization state prediction data at each stage in the Bi-LSTM and output a multi-period and multi-state hospitalization and hospitalization state prediction vector;
(6) the prediction model training module is used for constructing a softmax prediction function by utilizing the hospitalization state prediction vector; calculating a loss function of the output value of the softmax prediction function, and training the learning parameters of the Bi-LSTM by adopting a back propagation algorithm to complete the training of the model;
specifically, in the prediction model training module, the cross entropy is used as a loss function of the prediction model to realize the training of the model.
(7) And the feedback updating module is used for outputting the prediction result of the experimental sample set after the model training is finished, comparing the prediction result with the actual hospitalization and hospitalization behaviors, and feeding back and updating the weight value of the prediction data of the hospitalization and hospitalization state so as to continuously perfect the prediction of the hospitalization and hospitalization behaviors.
Specifically, in the feedback updating module, the predicted result and the actual result are compared, the weight value of attraction data of the hospital-disease state in the Attention mechanism is adjusted based on the interpretable feedback of the predicted result, the weight of each dimension in the prediction data of the hospitalization state is sorted according to the reverse order, then the data with the preset number of the weights before are selected, and the importance degree of the hospitalization behavior prediction of the hospitalization record is fed back and adjusted.
In another embodiment, the device for predicting hospitalization and hospitalization behavior of Bi-LSTM with time-varying attention improvement further comprises:
and the preprocessing module is used for preprocessing the mass medical insurance data before extracting the characteristic that the correlation degree with the hospitalizing behavior is greater than the preset correlation degree threshold value from the mass medical insurance data, and comprises data cleaning, missing data completion, data definition and standardized processing.
In another embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which program, when being executed by a processor, carries out the steps of the in-hospital medical practice prediction method as shown in fig. 1.
In another embodiment, a computer device is provided, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the hospitalization and hospitalization behavior prediction method as shown in fig. 1 when executing the program.
As will be appreciated by one skilled in the art, embodiments of the present disclosure may be provided as a method, system, or computer program product. Accordingly, the present disclosure may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present disclosure is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
Although the present disclosure has been described with reference to specific embodiments, it should be understood that the scope of the present disclosure is not limited thereto, and those skilled in the art will appreciate that various modifications and changes can be made without departing from the spirit and scope of the present disclosure.
Claims (10)
1. A time-varying attention-improving Bi-LSTM hospitalization behavior prediction method, comprising:
the mass medical insurance data are subjected to standardized processing, feature selection influencing hospitalizing factors is carried out based on filtering type selection, feature redundancy is eliminated, and features with the degree of correlation with hospitalizing behaviors larger than a preset degree of correlation threshold are extracted; constructing Bi-LSTM by using the extracted hospitalization behavior characteristics and the corresponding weights thereof to realize the hospitalization state prediction of the hospitalization information in the positive and negative directions;
updating the weight value of each hospitalization state prediction data in the Bi-LSTM by adopting an attention mechanism based on the hospitalization state prediction data acquired by the Bi-LSTM and in combination with the hospitalization statistical data of each hospital transmitted by the bottom layer;
forming a pathological time rule based on the bottom layer transmission data, and constructing a time adjustment function according to the change of the influence degree of hospitalization behavior characteristics along with the increase of the time distance;
respectively multiplying the updated weight values of the hospitalization and hospitalization state prediction data by using a time adjustment function, thereby updating the weight values of the hospitalization and hospitalization state prediction data at each stage in the Bi-LSTM and outputting a multi-period and multi-state hospitalization and hospitalization state prediction vector;
constructing a softmax prediction function by using the hospitalization and hospitalization state prediction vector; calculating a loss function of the output value of the softmax prediction function, and training the learning parameters of the Bi-LSTM by adopting a back propagation algorithm to complete the training of the model;
and after the model training is finished, outputting the prediction result of the experimental sample set, comparing the prediction result with the actual hospitalization behavior, and feeding back and updating the weight value of the prediction data of the hospitalization state, so that the prediction of the hospitalization behavior is continuously perfected.
2. The method of predicting hospitalization and hospitalization behaviors of time-varying attention-improving Bi-LSTM of claim 1, wherein before extracting the feature having correlation greater than the preset correlation threshold with hospitalization and hospitalization behaviors from the mass medical insurance data, further comprising:
and carrying out data preprocessing on the mass medical insurance data, including data cleaning, missing data completion, data definition and normalized processing.
3. The method for predicting hospitalization and hospitalization behaviors of time-varying attention-improving Bi-LSTM according to claim 2, wherein the step of extracting the feature having the correlation degree with the hospitalization and hospitalization behaviors larger than the preset correlation threshold from the mass medical insurance data comprises:
extracting medical insurance data samples similar to the same type and different types from the medical insurance data subjected to the standardized processing based on the characteristics, and calculating characteristic-related statistics;
and setting a related statistic threshold value as a preset relevance threshold value, and screening out features related to hospitalization and hospitalization behaviors.
4. The method as claimed in claim 1, wherein an Attention-varying improving Bi-LSTM hospitalization and hospitalization behavior prediction method is used to construct an Attention mechanism and a time adjustment function, so as to simplify the hidden hierarchy of Bi-LSTM, increase the utilization rate of the effective data at the bottom layer, simulate the expert discriminant rule, and further increase the interpretability of the neural network model.
5. The method of claim 1, wherein the predicted outcome and the actual outcome are compared, the weight values of the hospital-illness attraction data in the Attention mechanism are adjusted based on interpretable feedback of the predicted outcome, the weights of each dimension in the hospitalization information state prediction data are sorted in a reverse order, a preset amount of data with the weights higher than the previous weights are selected, and the importance of hospitalization record on hospitalization information behavior prediction is feedback adjusted.
6. A time-varying attention-improving Bi-LSTM hospitalization and hospitalization behavior prediction apparatus, comprising:
the characteristic extraction module is used for carrying out standardized processing on mass medical insurance data, carrying out characteristic selection influencing hospitalizing factors based on filtering type selection, eliminating characteristic redundancy and extracting characteristics of which the correlation degree with hospitalizing behaviors is greater than a preset correlation threshold;
the Bi-LSTM construction module is used for constructing the Bi-LSTM by utilizing the extracted hospitalization behavior characteristics and the corresponding weights thereof so as to realize the hospitalization state prediction of the hospitalization information in the positive and negative directions;
the weight updating module is used for updating the weight value of each hospitalization state prediction data in the Bi-LSTM by adopting an attention mechanism based on the hospitalization state prediction data acquired by the Bi-LSTM and combining the hospitalization statistical data of each hospital transmitted by the bottom layer;
the time adjustment function building module is used for forming a pathological time rule based on bottom layer transmission data and building a time adjustment function according to the change of the influence degree of hospitalization and hospitalization behavior characteristics along with the increase of time distance;
the state prediction output module is used for multiplying the updated weight values of the hospitalization and hospitalization state prediction data by using a time adjusting function, so as to update the weight values of the hospitalization and hospitalization state prediction data at each stage in the Bi-LSTM and output a multi-period and multi-state hospitalization and hospitalization state prediction vector;
the prediction model training module is used for constructing a softmax prediction function by utilizing the hospitalization state prediction vector; calculating a loss function of the output value of the softmax prediction function, and training the learning parameters of the Bi-LSTM by adopting a back propagation algorithm to complete the training of the model;
and the feedback updating module is used for outputting the prediction result of the test set after the model training is finished, comparing the prediction result with the actual hospitalization and hospitalization behaviors, and feeding back and updating the weight value of the prediction data of the hospitalization and hospitalization state so as to continuously perfect the prediction of the hospitalization and hospitalization behaviors.
7. The apparatus of claim 6, further comprising:
the preprocessing module is used for preprocessing the mass medical insurance data before extracting the characteristic that the correlation degree with the hospitalizing and hospitalizing behaviors is larger than a preset correlation degree threshold value from the mass medical insurance data, and comprises data cleaning, missing data completion, data definition and standardization processing;
or in the prediction model training module, the cross entropy is adopted as the loss function of the prediction model to realize the training of the model;
or in the feedback updating module, comparing the predicted result with the actual result, adjusting the weight value of each hospital-disease attraction data in the Attention mechanism based on the interpretable feedback of the predicted result, sequencing the weight of each dimension in the hospitalization state predicted data according to the reverse order, then selecting the data with the preset number of the weights before, and feeding back and adjusting the importance of the hospitalization record to the hospitalization behavior prediction.
8. The apparatus of claim 7, wherein the feature extraction module is further configured to:
extracting medical insurance data samples similar to the same type and different types from the medical insurance data subjected to the standardized processing based on the characteristics, and calculating characteristic-related statistics;
and setting a related statistic threshold value as a preset relevance threshold value, and screening out features related to hospitalization and hospitalization behaviors.
9. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the in-hospital medical practice prediction method according to any one of claims 1 to 5.
10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps in the hospitalization and hospitalization behavior prediction method according to any of claims 1-5 when executing the program.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910324147.9A CN110334843B (en) | 2019-04-22 | 2019-04-22 | Time-varying attention improved Bi-LSTM hospitalization and hospitalization behavior prediction method and device |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910324147.9A CN110334843B (en) | 2019-04-22 | 2019-04-22 | Time-varying attention improved Bi-LSTM hospitalization and hospitalization behavior prediction method and device |
Publications (2)
Publication Number | Publication Date |
---|---|
CN110334843A CN110334843A (en) | 2019-10-15 |
CN110334843B true CN110334843B (en) | 2020-06-09 |
Family
ID=68140165
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201910324147.9A Active CN110334843B (en) | 2019-04-22 | 2019-04-22 | Time-varying attention improved Bi-LSTM hospitalization and hospitalization behavior prediction method and device |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN110334843B (en) |
Families Citing this family (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110765359B (en) * | 2019-10-30 | 2022-09-16 | 北京速途网络科技股份有限公司 | New media content recommendation method and system |
CN111061700A (en) * | 2019-11-12 | 2020-04-24 | 山大地纬软件股份有限公司 | Hospitalizing migration scheme recommendation method and system based on similarity learning |
CN110881037A (en) * | 2019-11-19 | 2020-03-13 | 北京工业大学 | Network intrusion detection method and training method and device of model thereof, and server |
CN111028913A (en) * | 2019-11-29 | 2020-04-17 | 北京工业大学 | Hemodialysis treatment scheme aid decision-making method |
CN111461455B (en) * | 2020-04-16 | 2022-05-13 | 武汉大学 | Behavior prediction method based on association cycle attention mechanism |
CN112395328A (en) * | 2020-05-22 | 2021-02-23 | 南京大学 | Key attribute mining method based on attention mechanism |
CN111612254B (en) * | 2020-05-22 | 2022-12-23 | 中国科学院合肥物质科学研究院 | Road motor vehicle exhaust emission prediction method based on improved attention bidirectional long-short term memory network |
CN112016302B (en) * | 2020-08-03 | 2024-04-30 | 青岛国新健康产业科技有限公司 | Identification method and device for decomposing hospitalization behaviors, electronic equipment and storage medium |
CN111984503B (en) * | 2020-08-17 | 2023-12-01 | 网宿科技股份有限公司 | Method and device for identifying abnormal data of monitoring index data |
CN112101660B (en) * | 2020-09-15 | 2022-06-28 | 重庆交通大学 | Rainfall type landslide displacement prediction model and method based on staged attention mechanism |
CN112561728B (en) * | 2020-10-28 | 2023-07-11 | 西安交通大学 | Comprehensive energy consumption optimization method, medium and device based on attention mechanism LSTM |
CN113807562A (en) * | 2021-06-25 | 2021-12-17 | 中国海洋大学 | Prediction method of ocean surface temperature |
CN113781110B (en) * | 2021-09-07 | 2024-03-26 | 中国船舶重工集团公司第七0九研究所 | User behavior prediction method and system based on multi-factor weighted BI-LSTM learning |
CN116646067A (en) * | 2023-07-26 | 2023-08-25 | 四川互慧软件有限公司 | Real-time dynamic scheduling method and system for emergency medical rescue |
CN117079821B (en) * | 2023-10-12 | 2023-12-19 | 北京大学第三医院(北京大学第三临床医学院) | Patient hospitalization event prediction method |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR102033411B1 (en) * | 2016-08-12 | 2019-10-17 | 한국전자통신연구원 | Apparatus and Method for Recognizing speech By Using Attention-based Context-Dependent Acoustic Model |
CN108491680A (en) * | 2018-03-07 | 2018-09-04 | 安庆师范大学 | Drug relationship abstracting method based on residual error network and attention mechanism |
CN109493933B (en) * | 2018-08-08 | 2022-04-05 | 浙江大学 | Attention mechanism-based adverse cardiovascular event prediction device |
-
2019
- 2019-04-22 CN CN201910324147.9A patent/CN110334843B/en active Active
Also Published As
Publication number | Publication date |
---|---|
CN110334843A (en) | 2019-10-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110334843B (en) | Time-varying attention improved Bi-LSTM hospitalization and hospitalization behavior prediction method and device | |
CN109902222B (en) | Recommendation method and device | |
CN106778014B (en) | Disease risk prediction modeling method based on recurrent neural network | |
JP7305656B2 (en) | Systems and methods for modeling probability distributions | |
KR102190299B1 (en) | Method, device and program for predicting the prognosis of gastric cancer using artificial neural networks | |
Bukharov et al. | Development of a decision support system based on neural networks and a genetic algorithm | |
Ba-Alwi et al. | Comparative study for analysis the prognostic in hepatitis data: data mining approach | |
Chopra et al. | Recurrent neural networks with non-sequential data to predict hospital readmission of diabetic patients | |
US20210375441A1 (en) | Using clinical notes for icu management | |
Tirumala | Evolving deep neural networks using coevolutionary algorithms with multi-population strategy | |
CN115132358A (en) | Machine learning for multi-state models of disease | |
CN110335160B (en) | Medical care migration behavior prediction method and system based on grouping and attention improvement Bi-GRU | |
Deb et al. | An outcome based analysis on heart disease prediction using machine learning algorithms and data mining approaches | |
Sun et al. | A fuzzy brain emotional learning classifier design and application in medical diagnosis | |
Abd Ali et al. | Networks data transfer classification based on neural networks | |
Dadgar et al. | A hybrid method of feature selection and neural network with genetic algorithm to predict diabetes | |
WO2005048185A1 (en) | Transductive neuro fuzzy inference method for personalised modelling | |
CN111402953B (en) | Protein sequence classification method based on hierarchical attention network | |
Christodoulou et al. | Improving the performance of classification models with fuzzy cognitive maps | |
Chatterjee et al. | A bi-fold approach to detect and classify COVID-19 X-ray images and symptom auditor | |
Shah et al. | A review on big data practices in healthcare | |
Shanthakumari et al. | Deep Convolutional Neural Network Model for Classifying Alzheimer’s Disease | |
Park et al. | Medical Time-series Prediction With LSTM-MDN-ATTN | |
Busi et al. | A Hybrid Deep Learning Technique for Feature Selection and Classification of Chronic Kidney Disease. | |
Okamoto et al. | Evolutionary Acquisition of CNN Architecture by Thermodynamical Genetic Algorithm |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |