WO2021151355A1 - 基于强化学习模型的疾病排序方法、装置、设备及介质 - Google Patents

基于强化学习模型的疾病排序方法、装置、设备及介质 Download PDF

Info

Publication number
WO2021151355A1
WO2021151355A1 PCT/CN2020/135340 CN2020135340W WO2021151355A1 WO 2021151355 A1 WO2021151355 A1 WO 2021151355A1 CN 2020135340 W CN2020135340 W CN 2020135340W WO 2021151355 A1 WO2021151355 A1 WO 2021151355A1
Authority
WO
WIPO (PCT)
Prior art keywords
disease
suspected
patient
model
ranking
Prior art date
Application number
PCT/CN2020/135340
Other languages
English (en)
French (fr)
Inventor
唐蕊
Original Assignee
平安科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技（深圳）有限公司 filed Critical 平安科技（深圳）有限公司
Publication of WO2021151355A1 publication Critical patent/WO2021151355A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/70—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
    • Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Definitions

  • This application relates to the field of artificial intelligence technology, and in particular to a disease ranking method, device, equipment and medium based on a reinforcement learning model.
  • the auxiliary diagnosis technology in the clinical decision support system is usually realized through the establishment of auxiliary diagnosis models by machine learning or deep learning methods. That is, the patient's condition information is input into the auxiliary diagnosis model, and the auxiliary diagnosis model outputs a list of suspected diseases for the patient. The doctor can refer to the suspected disease list given by the auxiliary diagnosis model to make a reference diagnosis of the patient's condition, thereby realizing auxiliary diagnosis The model assists the doctor in diagnosis.
  • the existing auxiliary diagnosis model will support multiple diseases, and the performance of multiple diseases in the model is basically stable. According to the performance of multiple diseases in the model, multiple disease types are determined as dominant diseases. In some auxiliary diagnosis models, the dominant disease types and their corresponding disease performance are unchanged, so that doctors in various regions have a unified judgment standard when using the auxiliary diagnosis model.
  • the diagnosis performance of the auxiliary diagnosis model is not optimized enough, resulting in a disease output result that is different from the actual disease diagnosis situation in the local area. , The accuracy is reduced.
  • the present application provides a disease ranking method, device, equipment, and medium based on a reinforcement learning model to solve the problem in the prior art that the auxiliary diagnosis model does not consider the disease conditions in different regions, resulting in low accuracy of disease output results.
  • a disease ranking method based on reinforcement learning model including:
  • the disease ranking result is a result of ranking multiple suspected diseases according to the probability of obtaining each disease by the patient;
  • the preset weight model is a reinforcement learning model obtained by performing disease weight learning based on the disease diagnosis data of the region where the patient belongs;
  • the suspected disease ranking result of the patient is determined and output.
  • a disease ranking device based on a reinforcement learning model including:
  • the first obtaining module is used to obtain the patient's condition data and input the patient's condition data into the auxiliary diagnosis model;
  • the second acquisition module is configured to acquire the disease ranking result output by the auxiliary diagnosis model, where the disease ranking result is a result of ranking multiple suspected diseases according to the probability of acquiring each disease by the patient;
  • the first determining module is configured to determine the weights of the multiple suspected diseases in the region to which the patient belongs according to a preset weight model, where the preset weight model is obtained by learning disease weights based on the disease diagnosis data of the region to which the patient belongs Reinforcement learning model;
  • the update module is used to update the ranking result of the suspected diseases according to the weights of the multiple suspected diseases in the region where the patient belongs, so as to obtain the updated disease ranking results;
  • the second determining module is configured to determine the patient's suspected disease ranking result according to the updated disease ranking result, and output it.
  • a computer device includes a memory, a processor, and computer-readable instructions that are stored in the memory and can run on the processor, and the processor implements the following steps when the processor executes the computer-readable instructions:
  • the disease ranking result is a result of ranking multiple suspected diseases according to the probability of obtaining each disease by the patient;
  • the preset weight model is a reinforcement learning model obtained by performing disease weight learning based on the disease diagnosis data of the region to which the patient belongs;
  • the suspected disease ranking result of the patient is determined and output.
  • One or more readable storage media storing computer readable instructions, when the computer readable instructions are executed by one or more processors, the one or more processors execute the following steps:
  • the disease ranking result is a result of ranking multiple suspected diseases according to the probability of obtaining each disease by the patient;
  • the preset weight model is a reinforcement learning model obtained by performing disease weight learning based on the disease diagnosis data of the region to which the patient belongs;
  • the suspected disease ranking result of the patient is determined and output.
  • a preset weight model based on the disease diagnosis data of each region is obtained through training, and then the weight of each suspected disease in the patient's region is determined according to the preset weight model, and then the disease ranking results are re-sorted according to the weight of each suspected disease Sorting, based on the existing auxiliary diagnosis model, considers the actual disease conditions in different regions, so that the final result of the sorting of the suspected diseases is more optimized, thereby improving the accuracy of the output results of the suspected diseases.
  • FIG. 1 is a schematic diagram of an application environment of a disease ranking method based on a reinforcement learning model in an embodiment of the present application
  • FIG. 2 is a schematic flowchart of a disease ranking method based on a reinforcement learning model in an embodiment of the present application
  • FIG. 3 is a schematic diagram of an implementation flow of step S30 in FIG. 2 of the present application.
  • FIG. 4 is a schematic diagram of an implementation flow of step S40 in FIG. 2 of the present application.
  • FIG. 5 is a schematic diagram of an implementation flow of step S50 in FIG. 2 of the present application.
  • FIG. 6 is a schematic diagram of an acquisition process of a preset weight model in an embodiment of the present application.
  • FIG. 7 is a schematic diagram of an implementation flow of step S04 in FIG. 6 of the present application.
  • FIG. 8 is a schematic structural diagram of a disease ranking device based on a reinforcement learning model in an embodiment of the present application.
  • Fig. 9 is a schematic structural diagram of a computer device in an embodiment of the present application.
  • the disease ranking method based on the reinforcement learning model provided by the embodiment of the present application can be applied in the application environment as shown in FIG. 1, wherein the terminal device communicates with the server through the network.
  • the server obtains the patient's condition data in the terminal device, and inputs the patient's condition data into the auxiliary diagnosis model, and then obtains the disease ranking result output by the auxiliary diagnosis model.
  • the disease ranking result is based on the probability that the patient obtains each disease for multiple suspected diseases After sorting the results, the weights of multiple suspected diseases in the patient's area are determined according to the preset weight model.
  • the preset weight model is a reinforcement learning model obtained by disease weight learning based on the disease diagnosis data of the patient's area, and then based on multiple The weight of the suspected disease in the region where the patient belongs is updated to obtain the updated disease ranking result. Finally, the patient’s suspected disease ranking result is determined according to the updated disease ranking result and output to the terminal device, thereby improving Accuracy of output results for suspected diseases.
  • the terminal device can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices.
  • the server can be implemented as an independent server or a server cluster composed of multiple servers.
  • the auxiliary diagnosis model, the preset weight model, and the related data of model input and output are all stored in the blockchain network.
  • Blockchain is a new application mode of computer technology such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm.
  • Blockchain essentially a decentralized database, is a series of data blocks associated with cryptographic methods. Each data block contains a batch of network transaction information for verification. The validity of the information (anti-counterfeiting) and the generation of the next block.
  • the blockchain can include the underlying platform of the blockchain, the platform product service layer, and the application service layer.
  • the auxiliary diagnosis model, the preset weight model, that is, related data are stored in the blockchain network, which facilitates quick query and processing of the auxiliary diagnosis model, the preset weight model, and related data, and improves the processing speed.
  • a disease ranking method based on a reinforcement learning model is provided. Taking the method applied to the server in FIG. 1 as an example, the method includes the following steps:
  • the condition data is the patient's medical record data, including the patient's basic information, the patient's self-reported condition information, and examination data.
  • the basic information includes routine data such as the patient's age, region, gender, etc.
  • the examination data includes image data, image data, and so on.
  • S20 Obtain a disease ranking result output by the auxiliary diagnosis model, where the disease ranking result is a result of ranking multiple suspected diseases according to the probability of obtaining the disease by the patient.
  • the auxiliary diagnosis model will output the disease ranking results for the patient, that is, output the results of ranking multiple suspected diseases according to the probability of the patient getting the disease, so that the doctor can According to the disease ranking results output by the auxiliary diagnosis model, the diagnosis is assisted, and the disease acquired by the patient is finally determined.
  • the disease ranking results output by the auxiliary diagnosis model need to be obtained, so as to optimize the disease ranking results according to the weights of multiple suspected diseases in the patient's region, thereby improving The accuracy of the patient's disease output results, thereby improving the assistance to the doctor.
  • the preset weight model is a reinforcement learning model obtained by performing disease weight learning based on the disease diagnosis data of the region where the patient belongs.
  • the weights of multiple suspected diseases in the patient's region are determined according to the preset weight model, where the preset weight model is obtained by learning disease weights based on the disease diagnosis data of the patient's region Reinforcement learning model.
  • the patient's area can be the patient's long-term residence area, the patient's household registration area, or the patient's treatment area.
  • S40 Update the ranking result of the suspected diseases according to the weights of the multiple suspected diseases in the region where the patient belongs to obtain the updated disease ranking result.
  • the ranking results of the suspected diseases are updated according to the weights of multiple suspected diseases in the patient's area, and the number of suspected diseases is updated according to the updated probability of obtaining the disease.
  • the suspected diseases are re-sorted to obtain the updated disease sorting results, so that the updated disease sorting results have higher accuracy.
  • S50 Determine the patient's suspected disease ranking result according to the updated disease ranking result, and output it.
  • the output of the auxiliary diagnosis model's disease ranking results are obtained.
  • the weights of multiple suspected diseases in the patient's area are determined according to the preset weight model.
  • the disease ranking results are optimized and updated, which can automatically obtain more optimized and closer to the actual disease situation of each region.
  • the automated processing process of artificial intelligence + disease recognition is realized without manual participation. A better ranking result of suspected diseases can be obtained, which is convenient for doctors to use as a reference for subsequent diagnosis, thereby improving the accuracy of disease diagnosis.
  • This solution can be applied in the field of smart medical care to promote the construction of smart cities.
  • the patient's condition data is obtained, and the patient's condition data is input into the auxiliary diagnosis model, and then the disease ranking result output by the auxiliary diagnosis model is obtained, and then multiple suspects are determined according to the preset weight model
  • the weight of the disease in the region where the patient belongs and then update the ranking result of the suspected disease according to the weight of multiple suspected diseases in the region where the patient belongs to obtain the updated disease ranking result, and finally determine the patient's suspected disease according to the updated disease ranking result
  • Sorting results A preset weight model based on the disease diagnosis data of each region is obtained through training, and then the weight of each suspected disease in the patient's region is determined according to the preset weight model, and then the disease ranking results are re-ranked according to the weight of each suspected disease Based on the existing auxiliary diagnosis model, considering the actual disease conditions in different regions, the final result of the sorting of the suspected diseases is more optimized, thereby improving the accuracy of the output results of the suspected diseases.
  • step S30 that is, determining the weights of multiple suspected diseases in the region where the patient belongs according to the preset weight model, specifically includes the following steps:
  • S31 Use the state output by the preset weight model of the region to which the patient belongs as the weight of multiple dominant disease types in the region to which the patient belongs.
  • the auxiliary diagnosis model After obtaining the disease ranking results output by the auxiliary diagnosis model, it is necessary to obtain the preset weight model of the region to which the patient belongs, and use the output state of the preset weight model of the region to which the patient belongs as the status of multiple dominant diseases in the region to which the patient belongs.
  • the weight is used to update the disease ranking results of the auxiliary diagnosis model according to the weights of multiple dominant diseases in the patient's area.
  • S32 Determine the disease type of each suspected disease among multiple suspected diseases.
  • the multiple diseases are divided into different disease types based on the similarity of multiple suspected diseases, that is, the disease type of each suspected disease among the multiple suspected diseases is determined.
  • the purpose of categorizing diseases according to similarity is to reduce the impact on the performance of other similar diseases when the ranking results of the suspected diseases are updated according to the weight of the suspected diseases.
  • multiple suspected diseases include four diseases: disease A, disease B, disease C, and disease D.
  • disease B and disease D are different disease types
  • disease A and disease C are the same disease type, and are different from disease B and disease.
  • the disease type of disease D is different
  • the disease ranking result output by the auxiliary diagnosis model includes three disease types.
  • the process of determining multiple suspected diseases and disease types is merely illustrative. In other embodiments, the multiple suspected diseases may also be determined in other ways, which will not be repeated here.
  • S33 Determine whether the disease type of each suspected disease is multiple dominant disease types in the area where the patient belongs.
  • the weight of the dominant disease type is used as the weight corresponding to the suspected disease to obtain the weight of multiple suspected diseases.
  • the weight of the dominant disease type is taken as the corresponding suspected disease Weight to obtain the weight of multiple suspected diseases.
  • the weight of the disease type is the matched dominant disease type
  • the weight of the suspected disease corresponding to the disease type is also the weight of the dominant disease, so as to obtain the weight of multiple diseases. Due to the similarity between different suspected diseases, by dividing multiple suspected diseases into disease types, the weight of suspected diseases is set according to the type of disease, which reduces the impact of similar suspected diseases on each other. That is, similar diseases are considered uniformly, instead of considering a single disease, to ensure that the weights of similar diseases are updated at the same time, thereby reducing the impact of optimization of one disease on other diseases.
  • multiple suspected diseases include disease A, disease B, disease C, and disease D.
  • the disease types of disease A and disease C are disease type 1, the disease type of disease B is disease type 2, and disease D is disease type. It is disease type 3. If disease type 1 is the dominant disease type in the area where the patient belongs, the weight of the dominant disease type will be the weight of disease type 1, and the weight of disease A and disease C will be the weight of disease type 1; if disease type 2 If the disease is the dominant disease in the area where the patient belongs, the weight of the dominant disease is the weight of disease 1, and the weight of disease B is the weight of disease 1.
  • disease type 1 is the dominant disease type in the region to which the patient belongs or disease type 2 is the dominant disease type in the region to which the patient belongs is merely illustrative. In other embodiments, disease type 3 may also be the dominant disease type.
  • the probability of obtaining the suspected disease is not updated.
  • the state output by the preset weight model of the region is used as the weight of multiple dominant diseases in the region, and then the disease type of each suspected disease among the multiple suspected diseases is determined, and then the disease of each suspected disease is determined Whether the type is multiple dominant diseases in the area where the patient belongs, if the type of the suspected disease is multiple dominant diseases in the area where the patient belongs, the weight of the dominant disease is used as the weight corresponding to the suspected disease to obtain multiple suspected diseases It refines the process of determining the weight of multiple suspected diseases in the patient’s area based on the preset weight model. It also considers the disease according to its category, and uses the weight of the disease category as the weight of each corresponding disease, reducing the number of similar diseases. The impact of suspected diseases makes the weight more accurate, which in turn makes the subsequent update of the disease ranking results more accurate.
  • step S40 the ranking result of the suspected diseases is updated according to the weights of the multiple suspected diseases in the region where the patient belongs, which specifically includes the following steps:
  • S41 Determine the probability of obtaining each suspected disease according to the sorting result of the suspected disease.
  • the probability of obtaining each suspected disease is determined according to the result of the sorting of the suspected diseases. That is, the result of the ranking of suspected diseases includes multiple suspected diseases and the probability of obtaining each suspected disease, and the probability of obtaining each suspected disease is extracted from the result of the suspected disease ranking.
  • S42 Determine the product of the weight of the suspected disease in the area where the patient belongs and the probability of obtaining the suspected disease as the final probability of obtaining the suspected disease.
  • the product of the weight of the suspected disease in the area where the patient belongs and the probability of obtaining the suspected disease is determined as the final probability of obtaining the suspected disease.
  • the first column in Table 1 is the suspected disease and the probability of obtaining each suspected disease output by the auxiliary diagnosis model
  • the third and fourth columns are the type of suspected disease and the weight of the suspected disease
  • the fifth column is the suspected disease.
  • the updated probability of obtaining that is, the final probability of obtaining the suspected disease.
  • the ranking of multiple suspected diseases is updated according to the final acquisition probability of each suspected disease to obtain the updated disease ranking result. For example, a number of suspected diseases can be sorted in descending order of the probability of being obtained according to the size of the finally obtained probability, and then the updated disease ranking result can be obtained.
  • sorting the multiple suspected diseases in the descending order of the probability of obtaining is only exemplary.
  • the multiple suspected diseases may also be sorted in other ways, for example, Sort according to the average acquisition probability of the disease type, sort the different disease types according to the average acquisition probability of the disease type in descending order, and then sort the suspected diseases of the same disease category according to the acquisition probability of the suspected disease, so as to obtain The updated disease ranking results.
  • the probability of obtaining each suspected disease is determined according to the result of the sorting of the suspected diseases, and then the product of the weight of the suspected disease in the area where the patient belongs and the probability of obtaining the suspected disease is determined as the final probability of obtaining the suspected disease, and then The ranking of multiple suspected diseases is updated according to the final probability of each suspected disease, and the steps for updating the ranking results of suspected diseases based on the weight of multiple suspected diseases in the patient's area are refined.
  • step S50 that is, determining the patient's suspected disease ranking result according to the updated disease ranking result, specifically includes the following steps:
  • the acquisition probability of the suspected disease is determined in the updated disease ranking result, that is, the final acquisition probability after updating according to the weight is determined.
  • S52 Sort the suspected diseases from high to low according to the probability of obtaining the suspected diseases, and obtain a ranking list of the suspected diseases.
  • the suspected diseases are sorted from high to low according to the probability of obtaining the suspected disease, and a list of the suspected diseases is obtained.
  • S53 Select a preset number of suspected diseases and the probability of obtaining the suspected diseases in the list of suspected diseases as the result of the patient's suspected disease ranking.
  • the preset number is 10
  • select the top 10 suspected diseases and the probability of obtaining suspected diseases from the list of suspected diseases as the patient's suspected disease ranking result and output the probability of obtaining the top 10 suspected diseases and suspected diseases.
  • the final disease ranking result is clear at a glance, which is convenient for doctors to quickly browse and refer to, and then assist the doctor in diagnosing the patient's actual disease situation, and improve the output efficiency of the final disease ranking result.
  • the preset number of 10 is only an exemplary illustration. In other embodiments, the preset number may also be other values, which will not be repeated here.
  • the probability of obtaining the suspected disease is determined in the updated disease ranking result, and then the suspected diseases are sorted from high to low according to the probability of obtaining the suspected disease, and then the list of suspected diseases is obtained.
  • the pre-predetermined number of suspected diseases and the probability of obtaining suspected diseases are selected as the patient's suspected disease ranking result.
  • the steps to determine the patient's suspected disease ranking result according to the updated disease ranking result are refined, and the final result is improved.
  • the output efficiency of the disease ranking results makes the final disease ranking results clear at a glance, which is convenient for doctors to quickly browse and reference.
  • the preset weight model before determining the weights of multiple suspected diseases in the patient's region according to the preset weight model, it is also necessary to perform disease weight learning based on the disease diagnosis data of the patient's region to obtain the preset weight model. Set the weight model to obtain more accurate weights for multiple suspected diseases. As shown in Fig. 6, before step S30, the preset weight model is specifically obtained in the following manner:
  • S01 Determine the k dominant disease types in the region where the patient belongs, and the dominant disease types are multiple disease types in the region where the patient belongs that the frequency of occurrence of the disease is higher than the preset frequency.
  • the dominant disease type is the multiple disease types whose disease frequency is higher than the preset frequency in the region where the patient belongs
  • the dominant disease type is the disease type in the auxiliary diagnosis model, that is, determine In the region where the patient belongs, there are k disease types whose disease frequency is higher than the preset frequency, and the k disease types whose frequency is higher than the preset frequency are used as the dominant disease types to facilitate the training of the preset weight model.
  • S02 Define the weights of k dominant disease types as the state of the pre-training model, and the state is a k-dimensional vector.
  • the weight of the k dominant disease types is defined as the state of the pre-training model, where the state of the pre-training model is a k-dimensional vector.
  • the pre-training model may be a DQN (Deep Q-learning Network) model.
  • the pre-training model may also be other reinforcement learning models, which will not be repeated here.
  • the pre-training model is taken as an example for description.
  • S03 Input the k-dimensional vector into the neural network of the pre-training model to obtain the action of the pre-training model.
  • the vectors representing the weights of the k dominant diseases are input into the neural network of the DQN model as the actions of the DQN model. That is, for the k dominant disease types in the area where the patient belongs, the weight of each dominant disease type is increased or decreased, which is also represented by a k-dimensional vector, which is an action.
  • the state state is a k-dimensional vector that represents the weight of the current k-type disease
  • the action action is represented by a k-dimensional one-hot vector, for example, k is 3, and the three-dimensional vector of the action ([Disease Category 1 , Disease category 2, disease category 3]), the action's three-dimensional vector [0, 1, 0] indicates that the weight of disease category 2 has increased, and the action's three-dimensional vector [0, 0, -1] indicates that the weight of disease category 3 has decreased,
  • Each action has only one corresponding disease category changed, and the status update in the DQN model is based on the current status and action.
  • k being 3 is only an exemplary illustration, and in other embodiments, k may also be other values, which will not be repeated here.
  • S04 Determine the reward of the pre-training model according to the disease diagnosis data of the region where the patient belongs.
  • the reward of the pre-training model is determined according to the disease diagnosis data of the area where the patient belongs. Reward rewards play a role in the training process of the pre-training model, and the current state of the pre-training model is updated through rewards.
  • the disease diagnosis data of the patient's region includes the results of the auxiliary diagnosis model ranking the diagnosed patients' diseases.
  • the state is updated and changed.
  • the auxiliary diagnosis is based on the updated state.
  • the disease ranking results of the model are updated to obtain disease performance in different states. If the disease performance in the current state is improved, the reward is 1; if the disease performance in the current state remains unchanged, the reward is 0; If the disease performance is reduced, the reward is -1.
  • the determination of the reward is only illustrative. In other embodiments, the reward can also be set to others, which will not be repeated here.
  • S05 Adjust the state, action, and reward to perform weight learning on the pre-trained model to obtain a preset weight model.
  • the state, action and reward are constantly adjusted so that the loss function of the pre-training model no longer changes.
  • the state of the pre-training model is stable, indicating the performance of the pre-training model Compared with the disease diagnosis data of the region where the patient belongs, no changes will occur.
  • the steady-state pre-training model is used as the preset weight model.
  • the k-dimensional vector represented by the steady state is the pre-training model.
  • the output result of the weight model that is, the k-dimensional vector output by the preset weight model is the weight of k dominant diseases.
  • the dominant disease types are multiple disease types in the region where the patient belongs that the frequency of occurrence of the disease is higher than the preset frequency
  • the weight of the k dominant disease types is defined as The state of the pre-training model, the state is a k-dimensional vector, and then the k-dimensional vector is input into the neural network of the pre-training model to obtain the actions of the pre-training model, and then the pre-training model is determined according to the disease diagnosis data of the patient's region Reward, finally adjust the state, action and reward to learn the weight of the pre-trained model, obtain the preset weight model, clarify the process of obtaining the preset weight model, and train to obtain the preset weight model according to the disease diagnosis data of the patient's region, so that The preset weight model is close to the data situation of the patient's area, improves the accuracy of the preset weight model, and provides a basis for subsequent optimization of the disease ranking results of the auxiliary diagnosis model.
  • the disease diagnosis data of the region where the patient belongs includes the disease diagnosis results of multiple confirmed patients and the auxiliary diagnosis model's disease ranking results for multiple confirmed patients.
  • the disease diagnosis data determines the reward of the pre-training model, which specifically includes the following steps:
  • the auxiliary diagnosis model updates the disease ranking results of multiple confirmed patients to determine the updated disease results of multiple confirmed patients in each state.
  • the updated disease results are the updated disease ranking results of the confirmed patients The disease with the highest probability of being acquired later.
  • the disease diagnosis data of the region to which the patient belongs includes the disease diagnosis results of multiple confirmed patients and the auxiliary diagnosis model.
  • the disease ranking results of multiple confirmed patients In the process of updating the state of the pre-training model, it is necessary to obtain the dominant disease in each state. Then, according to the weight of the dominant disease in each state, the auxiliary diagnosis model is updated for the disease ranking results of multiple confirmed patients to obtain the updated ranking results of the disease ranking results for each state, and then determined according to the updated ranking results Updated disease results for multiple confirmed patients in each state. Among them, the updated disease result is the disease with the highest probability of obtaining after updating the disease ranking result of the confirmed patient.
  • S042 Determine the accuracy rate of the updated disease results of the multiple confirmed patients in each state according to the disease diagnosis results of the multiple confirmed patients, so as to obtain the accuracy rate of the disease results in each state.
  • the accuracy of the updated disease results of multiple confirmed patients in each state is determined according to the disease diagnosis results of multiple confirmed patients to obtain the accuracy of the disease results in each state .
  • the k-dimensional vector of the initial state of the pre-training model is the average accuracy rate of various dominant diseases, that is, the multiple disease ranking results of the auxiliary diagnosis model are updated in the initial state, and the multiple updated disease results obtained Average accuracy rate.
  • S043 Determine the reward of the next state in the pre-training model according to the accuracy of the disease results in the two states before and after.
  • the reward of the next state in the pre-training model is determined according to the accuracy rate of the disease result in the two states before and after.
  • accu before represents the accuracy rate of the disease result in the last state
  • accu now represents the accuracy rate of the disease result in the current state
  • the threshold is 0.01
  • the process of determining the reward by the preset training model is: if
  • the threshold of 0.01 is only an exemplary description. In other embodiments, the threshold may also be other values less than 0.01, which will not be repeated here.
  • the auxiliary diagnosis model is updated according to the weight of the dominant disease in each state for the disease ranking results of multiple confirmed patients to determine the updated disease results of multiple confirmed patients in each state, and then based on the multiple confirmed patients’
  • the disease diagnosis result determines the accuracy of the updated disease results of multiple diagnosed patients in each state to obtain the accuracy of the disease results in each state, and determine the next state of the pre-training model according to the accuracy of the disease results in the two states before and after.
  • Reward which refines the process of determining the reward of the pre-training model based on the disease diagnosis data of the patient’s region, provides a basis for the determination of the reward, and makes the pre-training model combined with the auxiliary diagnosis model output during the training process close to the diagnosed patient’s disease performance
  • the actual disease diagnosis result improves the accuracy of the preset weight model.
  • a disease ranking device based on a reinforcement learning model corresponds to the disease ranking method based on the reinforcement learning model in the above-mentioned embodiment in a one-to-one correspondence.
  • the device for sorting diseases based on the reinforcement learning model includes a first acquisition module 801, a second acquisition module 802, a first determination module 803, an update module 804, and a second determination module 805.
  • the detailed description of each functional module is as follows:
  • the first obtaining module 801 is used to obtain the patient's condition data and input the patient's condition data into the auxiliary diagnosis model;
  • the second obtaining module 802 is configured to obtain a disease ranking result output by the auxiliary diagnosis model, where the disease ranking result is a result of ranking multiple suspected diseases according to the probability of obtaining each disease by the patient;
  • the first determining module 803 is configured to determine the weights of the multiple suspected diseases in the region to which the patient belongs according to a preset weight model, where the preset weight model is to perform disease weight learning based on the disease diagnosis data of the region to which the patient belongs Reinforcement learning model obtained;
  • the update module 804 is configured to update the ranking result of the suspected diseases according to the weights of the multiple suspected diseases in the region to which the patient belongs, so as to obtain the updated disease ranking results;
  • the second determining module 805 is configured to determine the patient's suspected disease ranking result according to the updated disease ranking result, and output it.
  • the first determining module 803 is specifically configured to:
  • the weight of the dominant disease type is used as the weight of the corresponding suspected disease to obtain the weight of the multiple suspected diseases.
  • update module 804 is specifically configured to:
  • the ranking of the multiple suspected diseases is updated according to the final probability of each suspected disease.
  • the second determining module 805 is specifically configured to:
  • a preset number of the suspected diseases and the probability of obtaining the suspected diseases are selected from the suspected disease ranking list as the suspected disease ranking results of the patient.
  • the device for sorting diseases based on the reinforcement learning model further includes a model training module 806, and the model training module 806 is specifically configured to:
  • the state, the action, and the reward are adjusted to perform weight learning on the pre-training model to obtain the preset weight model.
  • the disease diagnosis data of the region to which the patient belongs includes disease diagnosis results of multiple confirmed patients and disease ranking results of the auxiliary diagnosis model for multiple confirmed patients, and the model training module 806 is specifically used to:
  • the auxiliary diagnosis model updates the disease ranking results of the multiple confirmed patients to determine the updated disease results of the multiple confirmed patients in each state, and the updated disease results are updating the disease results.
  • the reward of the next state in the pre-training model is determined according to the accuracy of the disease results in the two states before and after.
  • the various modules in the above-mentioned disease ranking device based on the reinforcement learning model can be implemented in whole or in part by software, hardware, and a combination thereof.
  • the above-mentioned modules may be embedded in the form of hardware or independent of the processor in the computer equipment, or may be stored in the memory of the computer equipment in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.
  • a computer device is provided.
  • the computer device may be a server, and its internal structure diagram may be as shown in FIG. 9.
  • the computer equipment includes a processor, a memory, a network interface, and a database connected through a system bus. Among them, the processor of the computer device is used to provide calculation and control capabilities.
  • the memory of the computer device includes a non-volatile storage medium and an internal memory.
  • the non-volatile storage medium stores an operating system, computer readable instructions, and a database.
  • the internal memory provides an environment for the operation of the operating system and computer-readable instructions in the non-volatile storage medium.
  • the database of the computer equipment is used to store auxiliary diagnosis models, preset weight models and disease ranking results.
  • the network interface of the computer device is used to communicate with an external terminal through a network connection.
  • the computer-readable instructions are executed by the processor to realize a disease ranking method based on a reinforcement learning model.
  • a computer device including a memory, a processor, and computer-readable instructions stored on the memory and capable of running on the processor, and the processor implements the following steps when the processor executes the computer-readable instructions:
  • the disease ranking result is a result of ranking multiple suspected diseases according to the probability of obtaining each disease by the patient;
  • the preset weight model is a reinforcement learning model obtained by performing disease weight learning based on the disease diagnosis data of the region to which the patient belongs;
  • the suspected disease ranking result of the patient is determined and output.
  • a computer-readable storage medium on which computer-readable instructions are stored, and when the computer-readable instructions are executed by a processor, the following steps are implemented:
  • the disease ranking result is a result of ranking multiple suspected diseases according to the probability of obtaining each disease by the patient;
  • the preset weight model is a reinforcement learning model obtained by performing disease weight learning based on the disease diagnosis data of the region where the patient belongs;
  • the suspected disease ranking result of the patient is determined and output.
  • Non-volatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • Volatile memory may include random access memory (RAM) or external cache memory.
  • RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Channel (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Data Mining & Analysis (AREA)
  • Biomedical Technology (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Pathology (AREA)
  • Epidemiology (AREA)
  • Theoretical Computer Science (AREA)
  • Primary Health Care (AREA)
  • Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Medical Treatment And Welfare Office Work (AREA)

Abstract

一种基于强化学习模型的疾病排序方法、装置、设备及介质，所述方法包括：获取病人的病情数据，并将病人的病情数据输入辅助诊断模型（S10）；获取辅助诊断模型输出的疾病排序结果（S20）；根据预设权重模型确定多个疑似疾病在病人所属地区的权重（S30）；根据多个疑似疾病在病人所属地区的权重对疑似疾病排序结果进行更新，以获得更新后的疾病排序结果（S40）；根据更新后的疾病排序结果确定病人的疑似疾病排序结果并进行输出（S50）。所述方法在已有的辅助诊断模型的基础上，考虑了不同地区的实际疾病情况，使得最终获得的疑似疾病排序结果更加优化，从而提高了疑似疾病输出结果准确性。

Description

基于强化学习模型的疾病排序方法、装置、设备及介质
本申请要求于2020年09月07日提交中国专利局、申请号为202010929683.4，发明名称“基于强化学习模型的疾病排序方法、装置、设备及介质”的中国发明专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及人工智能技术领域，尤其涉及一种基于强化学习模型的疾病排序方法、装置、设备及介质。
背景技术
随着人工智能技术的快速发展，在临床决策支持***中的辅助诊断技术通常通过机器学习或深度学习方法建立辅助诊断模型来实现。即，将病人的病情信息输入辅助诊断模型，辅助诊断模型输出针对病人的疑似疾病列表，医生通过参考辅助诊断模型给出的疑似疾病列表，可对病人的病情进行参考性诊断，从而实现辅助诊断模型对医生诊断的辅助。
一般来说，已有的辅助诊断模型会支持多个疾病，多个疾病在模型中的性能基本稳定，根据多种疾病在模型中的性能确定了多个疾病种类为优势病种，即在已有的辅助诊断模型中，优势病种及其对应的疾病性能是不变的，以使得各地区医生在使用辅助诊断模型时具有统一的判断标准。
技术问题
但是，发明人发现，不同地区中的病人对不同疾病的获得概率不同，即在不同的地区中的优势病种(在当地出现频率较高的多个疾病种类)不同，而现有的辅助诊断模型中，各个疾病在辅助诊断模型中的性能是统一确定的，未考虑不同地区的优势病种需求，辅助诊断模型的诊断性能不够优化，导致获得的疾病输出结果与当地实际的疾病诊断情况不同，准确性降低。
技术解决方案
本申请提供一种基于强化学习模型的疾病排序方法、装置、设备及介质，以解决现有技术中，辅助诊断模型未考虑不同地区的疾病情况，导致疾病输出结果准确性较低的问题。
一种基于强化学习模型的疾病排序方法，包括：
获取病人的病情数据，并将所述病人的病情数据输入辅助诊断模型；
获取所述辅助诊断模型输出的疾病排序结果，所述疾病排序结果为根据所述病人获得各疾病的概率大小对多个疑似疾病进行排序的结果；
根据预设权重模型确定所述多个疑似疾病在所述病人所属地区的权重，所述预设权重 模型为根据所述病人所属地区的疾病诊断数据进行疾病权重学习获得的强化学习模型；
根据所述多个疑似疾病在所述病人所属地区的权重对所述疑似疾病排序结果进行更新，以获得更新后的疾病排序结果；
根据所述更新后的疾病排序结果确定所述病人的疑似疾病排序结果，并进行输出。
一种基于强化学习模型的疾病排序装置，包括：
第一获取模块，用于获取病人的病情数据，并将所述病人的病情数据输入辅助诊断模型；
第二获取模块，用于获取所述辅助诊断模型输出的疾病排序结果，所述疾病排序结果为根据所述病人获得各疾病的概率大小对多个疑似疾病进行排序的结果；
第一确定模块，用于根据预设权重模型确定所述多个疑似疾病在所述病人所属地区的权重，所述预设权重模型为根据所述病人所属地区的疾病诊断数据进行疾病权重学习获得的强化学习模型；
更新模块，用于根据所述多个疑似疾病在所述病人所属地区的权重对所述疑似疾病排序结果进行更新，以获得更新后的疾病排序结果；
第二确定模块，用于根据所述更新后的疾病排序结果确定所述病人的疑似疾病排序结果，并进行输出。
一种计算机设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机可读指令，所述处理器执行所述计算机可读指令时实现如下步骤：
获取病人的病情数据，并将所述病人的病情数据输入辅助诊断模型；
获取所述辅助诊断模型输出的疾病排序结果，所述疾病排序结果为根据所述病人获得各疾病的概率大小对多个疑似疾病进行排序的结果；
根据预设权重模型确定所述多个疑似疾病在所述病人所属地区的权重，所述预设权重模型为根据所述病人所属地区的疾病诊断数据进行疾病权重学习获得的强化学习模型；
根据所述多个疑似疾病在所述病人所属地区的权重对所述疑似疾病排序结果进行更新，以获得更新后的疾病排序结果；
根据所述更新后的疾病排序结果确定所述病人的疑似疾病排序结果，并进行输出。
一个或多个存储有计算机可读指令的可读存储介质，所述计算机可读指令被一个或多个处理器执行时，使得所述一个或多个处理器执行如下步骤：
获取病人的病情数据，并将所述病人的病情数据输入辅助诊断模型；
获取所述辅助诊断模型输出的疾病排序结果，所述疾病排序结果为根据所述病人获得各疾病的概率大小对多个疑似疾病进行排序的结果；
根据预设权重模型确定所述多个疑似疾病在所述病人所属地区的权重，所述预设权重模型为根据所述病人所属地区的疾病诊断数据进行疾病权重学习获得的强化学习模型；
根据所述多个疑似疾病在所述病人所属地区的权重对所述疑似疾病排序结果进行更新，以获得更新后的疾病排序结果；
根据所述更新后的疾病排序结果确定所述病人的疑似疾病排序结果，并进行输出。
有益效果
本申请中，通过训练获得基于各地区的疾病诊断数据的预设权重模型，然后根据预设权重模型确定各疑似疾病在病人所属地区的权重，进而根据各疑似疾病的权重对疾病排序结果进行重新排序，在已有的辅助诊断模型的基础上，考虑了不同地区的实际疾病情况，使得最终获得的疑似疾病排序结果更加优化，从而提高了疑似疾病输出结果准确性。
本申请的一个或多个实施例的细节在下面的附图和描述中提出，本申请的其他特征和优点将从说明书、附图以及权利要求变得明显。
附图说明
为了更清楚地说明本申请实施例的技术方案，下面将对本申请实施例的描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1是本申请一实施例中基于强化学习模型的疾病排序方法的一应用环境示意图；
图2是本申请一实施例中基于强化学习模型的疾病排序方法的一流程示意图；
图3是本申请图2中步骤S30的一实现流程示意图；
图4是本申请图2中步骤S40的一实现流程示意图；
图5是本申请图2中步骤S50的一实现流程示意图；
图6是本申请一实施例中预设权重模型的一获取流程示意图；
图7是本申请图6中步骤S04的一实现流程示意图；
图8是本申请一实施例中基于强化学习模型的疾病排序装置的一结构示意图；
图9是本申请一实施例中计算机设备的一结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本申请实施例提供的基于强化学习模型的疾病排序方法，可应用在如图1的应用环境中，其中，终端设备通过网络与服务器进行通信。服务器通过获取终端设备中病人的病情数据，并将病人的病情数据输入辅助诊断模型，再获取辅助诊断模型输出的疾病排序结果，疾病排序结果为根据病人获得各疾病的概率大小对多个疑似疾病进行排序的结果，然后根据预设权重模型确定多个疑似疾病在病人所属地区的权重，预设权重模型为根据病人所属地区的疾病诊断数据进行疾病权重学习获得的强化学习模型，进而根据多个疑似疾病在病人所属地区的权重对疑似疾病排序结果进行更新，以获得更新后的疾病排序结果，最后根 据更新后的疾病排序结果确定病人的疑似疾病排序结果，并输出至终端设备，从而提高了疑似疾病输出结果准确性。
其中，终端设备可以但不限于各种个人计算机、笔记本电脑、智能手机、平板电脑和便携式可穿戴设备。服务器可以用独立的服务器或者是多个服务器组成的服务器集群来实现。
在本实施例中，辅助诊断模型、预设权重模型以及模型输入和输出的相关数据均保存在区块链网络中。区块链是分布式数据存储、点对点传输、共识机制、加密算法等计算机技术的新型应用模式。区块链(Blockchain)，本质上是一个去中心化的数据库，是一串使用密码学方法相关联产生的数据块，每一个数据块中包含了一批次网络交易的信息，用于验证其信息的有效性(防伪)和生成下一个区块。区块链可以包括区块链底层平台、平台产品服务层以及应用服务层。本实施例中将辅助诊断模型、预设权重模型即相关数据保存在区块链网络，便于对辅助诊断模型、预设权重模型和相关数据进行快速查询、处理，提高处理速度。
在一实施例中，如图2所示，提供一种基于强化学习模型的疾病排序方法，以该方法应用在图1中的服务器为例进行说明，包括如下步骤：
S10：获取病人的病情数据，并将病人的病情数据输入辅助诊断模型。
获取病人的病情数据，并将病人的病情数据输入辅助诊断模型。其中，病情数据为病人的病历数据，包括病人的基本信息、病人自述的病情信息和检查数据。其中，基本信息包括病人的年龄、所属地区、性别等常规数据，检查数据包括影像数据、图像数据等。
S20：获取辅助诊断模型输出的疾病排序结果，疾病排序结果为根据病人获得疾病概率的大小对多个疑似疾病进行排序的结果。
在传统方法中，在将病人的病情数据输入辅助诊断模型之后，辅助诊断模型会输出针对病人的疾病排序结果，即输出根据病人获得疾病概率的大小对多个疑似疾病进行排序的结果，以便医生根据辅助诊断模型输出的疾病排序结果辅助诊断，最终确定病人获得的疾病。而在本实施例中，在辅助诊断模型输出的疾病排序结果之后，需要获取辅助诊断模型输出的疾病排序结果，以根据多个疑似疾病在病人所属地区的权重对疾病排序结果进行优化，进而提高病人疾病输出结果的准确性，从而提高对医生的辅助作用。
S30：根据预设权重模型确定多个疑似疾病在病人所属地区的权重，预设权重模型为根据病人所属地区的疾病诊断数据进行疾病权重学习获得的强化学习模型。
在获取辅助诊断模型输出的疾病排序结果之后，根据预设权重模型确定多个疑似疾病在病人所属地区的权重，其中，预设权重模型为根据病人所属地区的疾病诊断数据进行疾病权重学习获得的强化学习模型。病人所属地区可以是病人的长居地区，也可以病人的户籍地区，还可以是病人的就诊地区。
S40：根据多个疑似疾病在病人所属地区的权重对疑似疾病排序结果进行更新，以获得更新后的疾病排序结果。
在根据预设权重模型确定多个疑似疾病在病人所属地区的权重之后，根据多个疑似疾病在病人所属地区的权重对疑似疾病排序结果进行更新，根据更新后的获得疾病概率的大小对多个疑似疾病进行重新排序，以获得更新后的疾病排序结果，使得更新后的疾病排序结果具有更高的准确性。
S50：根据更新后的疾病排序结果确定病人的疑似疾病排序结果，并进行输出。
在获得更新后的疾病排序结果之后，根据更新后的疾病排序结果确定病人的疑似疾病排序结果，并将病人的疑似疾病排序结果输出，以便医生在准确性更高的疑似疾病排序结果的辅助下对病人的疾病进行诊断。
在本实施例中，通过将病人的病情数据输入辅助诊断模型，获得辅助诊断模型的输出的疾病排序结果，在此基础上，根据预设权重模型确定多个疑似疾病在病人所属地区的权重，并根据多个疑似疾病的权重对疾病排序结果进行优化更新，可自动获得更加优化、贴近各地区实际疾病情况的疑似疾病排序结果，实现了人工智能+疾病识别的自动化处理过程，无需人工参与即可获得较优的疑似疾病排序结果，便于医生后续进行诊断时作为参考，从而提高了疾病诊断的准确性。本方案可应用于智慧医疗领域中，从而推动智慧城市的建设。
上述基于强化学习模型的疾病排序方法中，通过获取病人的病情数据，并将病人的病情数据输入辅助诊断模型，再获取辅助诊断模型输出的疾病排序结果，然后根据预设权重模型确定多个疑似疾病在病人所属地区的权重，进而根据多个疑似疾病在病人所属地区的权重对疑似疾病排序结果进行更新，以获得更新后的疾病排序结果，最后根据更新后的疾病排序结果确定病人的疑似疾病排序结果；通过训练获得基于各地区的疾病诊断数据的预设权重模型，然后根据预设权重模型确定各疑似疾病在病人所属地区的权重，进而根据各疑似疾病的权重对疾病排序结果进行重新排序，在已有的辅助诊断模型的基础上，考虑了不同地区的实际疾病情况，使得最终获得的疑似疾病排序结果更加优化，从而提高了疑似疾病输出结果准确性。
在一实施例中，如图3所示，步骤S30中，即根据预设权重模型确定多个疑似疾病在病人所属地区的权重，具体包括如下步骤：
S31：将病人所属地区的预设权重模型输出的状态作为病人所属地区中多个优势病种的权重。
在获取辅助诊断模型输出的疾病排序结果之后，需要获取训练好的病人所属地区的预设权重模型，并将病人所属地区的预设权重模型输出的状态作为病人所属地区中多个优势病种的权重，以根据病人所属地区中多个优势病种的权重对辅助诊断模型的疾病排序结果进行更新。
S32：确定多个疑似疾病中各疑似疾病的疾病种类。
在确定病人所属地区中多个优势病种的权重之后，根据多个疑似疾病的疾病相似性将多个疾病划分为不同疾病种类，即确定多个疑似疾病中各疑似疾病的疾病种类。将疾病按 照相似性分类是为了在后续根据疑似疾病的权重更新疑似疾病的排序结果时，减少对其他相似疾病种类的疾病的性能的影响。
例如，多个疑似疾病包括疾病A、疾病B、疾病C和疾病D四种疾病，其中，疾病B与疾病D为不同的疾病种类，疾病A和疾病C为同一疾病种类，且与疾病B和疾病D的疾病种类不同，则辅助诊断模型输出的疾病排序结果中包括三个疾病种类。
本实施例中，多个疑似疾病和疾病种类确定过程仅为示例性说明，在其他实施例中，还可以通过其他方式确定多个疑似疾病的疾病种类，在此不再赘述。
S33：确定各疑似疾病的疾病种类是否为病人所属地区的多个优势病种。
在确定各疑似疾病的疾病种类之后，确定各疑似疾病的疾病种类是否为病人所属地区的多个优势病种，以根据确定结果确定各疾病的权重。
S34：若疑似疾病的疾病种类为病人所属地区的多个优势病种，则将优势病种的权重作为对应疑似疾病的权重，以获得多个疑似疾病的权重。
在确定各疑似疾病的疾病种类是否为病人所属地区的多个优势病种之后，若疑似疾病的疾病种类为病人所属地区的多个优势病种，则将优势病种的权重作为对应疑似疾病的权重，以获得多个疑似疾病的权重。
在确定各疑似疾病的疾病种类是否为病人所属地区的多个优势病种之后，若疑似疾病的疾病种类为病人所属地区的多个优势病种，则疾病种类的权重则为匹配的优势病种的权重，对应的，疾病种类对应的疑似疾病的权重也为该优势病种的权重，从而获得多个疾病的权重。由于不同疑似疾病之间存在相似性，通过将多个疑似疾病进行疾病种类划分，对疑似疾病按照疾病种类设置权重，减少了相似疑似疾病之间对彼此的影响。即对相似的疾病统一进行考虑，而不是对单个疾病进行考虑，保证相似的疾病的权重都同同时更新，从而减少了对某一疾病进行优化导致对其他疾病的影响。
例如，多个疑似疾病包括疾病A、疾病B、疾病C和疾病D四种疾病，疾病A和疾病C的疾病种类为病种1，疾病B的疾病种类为病种2，疾病D为疾病种类为病种3，若病种1为病人所属地区的优势病种，则将优势病种的权重为病种1的权重，疾病A和疾病C的权重为病种1的权重；若病种2为病人所属地区的优势病种，则将优势病种的权重为病种1的权重，疾病B的权重为病种1的权重。
本实施例中，病种1为病人所属地区的优势病种或者病种2为病人所属地区的优势病种仅为示例性说明，在其他实施例中，病种3也可以为优势病种。
在确定各疑似疾病的疾病种类是否为病人所属地区的多个优势病种之后，若疑似疾病的疾病种类不为病人所属地区的多个优势病种，则不对疑似疾病的获得概率进行更新。
本实施例中，通过将所属地区的预设权重模型输出的状态作为所属地区中多个优势病种的权重，再确定多个疑似疾病中各疑似疾病的疾病种类，进而确定各疑似疾病的疾病种类是否为病人所属地区的多个优势病种，若疑似疾病的疾病种类为病人所属地区的多个优势病种，则将优势病种的权重作为对应疑似疾病的权重，以获得多个疑似疾病的权重，细 化了根据预设权重模型确定多个疑似疾病在病人所属地区的权重的过程，还通过对疾病按照类别进行考虑，将疾病类别的权重作为各相应疾病的权重，减少了对相似疑似疾病的影响，从而使得权重准确性更高，进而使得后续更新的疾病排序结果具有更高的准确性。
在一实施例中，如图4所示，步骤S40中，即根据多个疑似疾病在病人所属地区的权重对疑似疾病排序结果进行更新，具体包括如下步骤：
S41：根据疑似疾病排序结果确定各疑似疾病的获得概率。
在根据预设权重模型确定多个疑似疾病在病人所属地区的权重之后，根据疑似疾病排序结果确定各疑似疾病的获得概率。即疑似疾病排序结果包括多个疑似疾病和各疑似疾病的获得概率，在疑似疾病排序结果中提取出各疑似疾病的获得概率。
S42：确定疑似疾病在病人所属地区的权重与疑似疾病的获得概率之间的乘积，以作为疑似疾病的最终获得概率。
在根据疑似疾病排序结果确定各疑似疾病的获得概率之后，确定疑似疾病在病人所属地区的权重与疑似疾病的获得概率之间的乘积，以作为疑似疾病的最终获得概率。
例如，表1中的第一列为辅助诊断模型输出的疑似疾病和各疑似疾病的获得概率，第三列和第四列为疑似疾病的疾病种类和疑似疾病的权重，第五列为疑似疾病更新后的获得概率，即疑似疾病的最终获得概率。
表1
Figure PCTCN2020135340-appb-000001
从表1可知，根据疑似疾病的权重对辅助诊断模型输出的各疑似疾病获得概率之后，部分疑似疾病的获得概率发生了变化，获得概率最高的疑似疾病从疾病3变为了疾病2，使得更新后的结果更贴近病人所属地区的实际情况。
S43：根据各疑似疾病的最终获得概率对多个疑似疾病的排序进行更新。
在确定疑似疾病的最终获得概率之后，根据各疑似疾病的最终获得概率对多个疑似疾 病的排序进行更新，以获得更新后的疾病排序结果。例如，可以根据最终获得概率的大小，按照获得概率从大到小的顺序对多个疑似疾病进行排序，进而获得更新后的疾病排序结果。
本实施例中，按照获得概率从大到小的顺序对多个疑似疾病进行排序仅为示例性说明，在其他实施例中，还可以以其他的方式对多个疑似疾病进行排序，例如，可以根据疾病种类的平均获得概率进行排序，按照疾病种类的平均获得概率由大到小的顺序对不同的疾病种类进行排序，然后按照疑似疾病的获得概率对同一疾病类别的疑似疾病进行排序，从而获得更新后的疾病排序结果。
本实施例中，通过根据疑似疾病排序结果确定各疑似疾病的获得概率，再确定疑似疾病在病人所属地区的权重与疑似疾病的获得概率之间的乘积，以作为疑似疾病的最终获得概率，然后根据各疑似疾病的最终获得概率对多个疑似疾病的排序进行更新，细化了根据多个疑似疾病在病人所属地区的权重对疑似疾病排序结果进行更新的步骤。
在一实施例中，如图5所示，步骤S50中，即根据更新后的疾病排序结果确定病人的疑似疾病排序结果，具体包括如下步骤：
S51：在更新后的疾病排序结果中确定疑似疾病的获得概率。
在获得更新后的疾病排序结果之后，在更新后的疾病排序结果中确定疑似疾病的获得概率，即确定根据权重进行更新后的最终获得概率。
S52：根据疑似疾病的获得概率的大小对疑似疾病进行由高到低的排序，获得疑似疾病排序列表。
在确定疑似疾病的获得概率之后，根据疑似疾病的获得概率的大小对疑似疾病进行由高到低的排序，获得疑似疾病排序列表。
S53：在疑似疾病排序列表中选取前预设数量个疑似疾病和疑似疾病的获得概率作为病人的疑似疾病排序结果。
在获得疑似疾病排序列表之后，在疑似疾病排序列表中选取前预设数量个疑似疾病和疑似疾病的获得概率作为病人的疑似疾病排序结果。
例如，预设数量为10，则在疑似疾病排序列表中选取前10个疑似疾病和疑似疾病的获得概率作为病人的疑似疾病排序结果，以将前10个疑似疾病和疑似疾病的获得概率输出，使得最终的疾病排序结果一目了然，便于医生的快速浏览和参考，进而辅助医生的诊断病人的实际患病情况，提高了最终的疾病排序结果的输出效率。
本实施例中，预设数量为10仅为示例性说明，在其他实施例中，预设数量还可以是其他数值，在此不再赘述。
本实施例中，通过在更新后的疾病排序结果中确定疑似疾病的获得概率，再根据疑似疾病的获得概率的大小对疑似疾病进行由高到低的排序，然后获得疑似疾病排序列表，最后在疑似疾病排序列表中选取前预设数量个疑似疾病和疑似疾病的获得概率作为病人的疑似疾病排序结果，细化了根据更新后的疾病排序结果确定病人的疑似疾病排序结果的步骤，提高了最终的疾病排序结果的输出效率，使得最终的疾病排序结果一目了然，便于医 生的快速浏览和参考。
在一实施例中，在根据预设权重模型确定多个疑似疾病在病人所属地区的权重之前，还需要根据病人所属地区的疾病诊断数据进行疾病权重学习以获得预设权重模型，进而才能根据预设权重模型获得更加准确的多个疑似疾病权重。如图6所示，步骤S30之前，预设权重模型具体通过如下方式获取：
S01：确定病人所属地区的k个优势病种，优势病种为病人所属地区中疾病出现频率高于预设频率的多个疾病种类。
确定病人所属地区的k个优势病种，其中，优势病种为病人所属地区中疾病出现频率高于预设频率的多个疾病种类，且优势病种为辅助诊断模型中的疾病种类，即确定病人所属地区中疾病出现频率高于预设频率的k个疾病种类，并将出现频率高于预设频率的k个疾病种类作为优势病种，以便于训练预设权重模型。
S02：将k个优势病种的权重定义为预训练模型的状态，状态为k维的向量。
在确定病人所属地区的k个优势病种之后，将k个优势病种的权重定义为预训练模型的状态，其中，预训练模型的状态为一个k维的向量。
预训练模型可以是一个DQN(Deep Q-learning Network)模型，在其他实施例，预训练模型还可以是其他强化学习模型，在此不再赘述。本实施例中以预训练模型为例进行说明。
S03：将k维的向量输入预训练模型的神经网络中，以获得预训练模型的动作。
在确定预训练模型的动作之后，将表示k个优势病种的权重的向量输入DQN模型的神经网络中，以作为DQN模型的动作。即对于病人所属地区的k个优势病种，每一个优势病种的权重增加或减少，同样用k维的向量表示，即为动作。
例如，在DQN模型中，状态state为一个k维向量，表示当前k类疾病的权重；动作action，用k维的one-hot向量表示，例如k为3，动作的三维向量([疾病类别1，疾病类别2，疾病类别3])，动作的三维向量[0，1，0]表示疾病类别2的权重增加，动作的三维向量[0，0，-1]表示疾病类别3的权重减少，每个动作只有1个对应的疾病类别有变化，DQN模型中状态的更新是根据当前的状态和动作进行的。
本实施例中，k为3仅为示例性说明，在其他实施例中，k还可以是其他数值，在此不再赘述。
S04：根据病人所属地区的疾病诊断数据确定预训练模型的奖励。
根据病人所属地区的疾病诊断数据确定预训练模型的奖励。奖励reward是在预训练模型发训练过程中起作用的，通过奖励来更新预训练模型当前的状态。
例如，病人所属地区的疾病诊断数据包括了辅助诊断模型对确诊病人的疾病排序结果，在训练预训练模型的过程中，状态一种在更新变化，每次状态更新后，根据更新状态对辅助诊断模型的疾病排序结果进行更新，获得不同状态下的疾病性能，若当前状态下的疾病性能提高，则奖励为1；若当前状态下的疾病性能不变，则奖励为0；若当前状态下的疾 病性能降低，则奖励为-1。
本实施例中，奖励的确定仅为示例性说明，在其他实施例中，还可以将奖励设置为其他在此不再赘述。
S05：调整状态、动作和奖励以对预训练模型进行权重学习，获得预设权重模型。
在对预训练模型进行权重学习放入过程中，不断地调整状态、动作和奖励，以使得预训练模型的损失函数不再变化，此时预训练模型的状态达到稳定，表示预训练模型的性能与病人所属地区的疾病诊断数据相比不再发生变化，预训练模型训练完成，则将稳定状态下的预训练模型作为预设权重模型，此时，稳定状态所表示的k维的向量为预设权重模型的输出结果，即预设权重模型输出的k维的向量为k个优势病种的权重。
本实施例中，通过确定病人所属地区的k个优势病种，优势病种为病人所属地区中疾病出现频率高于预设频率的多个疾病种类，并将k个优势病种的权重定义为预训练模型的状态，状态为k维的向量，然后将k维的向量输入预训练模型的神经网络中，以获得预训练模型的动作，再根据病人所属地区的疾病诊断数据确定预训练模型的奖励，最后调整状态、动作和奖励以对预训练模型进行权重学习，获得预设权重模型，明确了获取预设权重模型的过程，根据病人所属地区的疾病诊断数据训练获得预设权重模型，使得预设权重模型贴近病人所属地区的数据情况，提高了预设权重模型的准确性，为后续对辅助诊断模型的疾病排序结果进行优化提供了基础。
在一实施例中，病人所属地区的疾病诊断数据包括多个确诊病人的疾病诊断结果和辅助诊断模型针对多个确诊病人的疾病排序结果，如图7所示，步骤S04中，即根据病人所属地区的疾病诊断数据确定预训练模型的奖励，具体包括如下步骤：
S041：根据各状态下优势病种的权重更新辅助诊断模型针对多个确诊病人的疾病排序结果，以确定各状态下多个确诊病人的更新疾病结果，更新疾病结果为更新确诊病人的疾病排序结果后的获得概率最高的疾病。
病人所属地区的疾病诊断数据包括多个确诊病人的疾病诊断结果和辅助诊断模型针对多个确诊病人的疾病排序结果，在更新预训练模型的状态的过程中，需要获取各状态下优势病种的权重，然后根据各状态下优势病种的权重对辅助诊断模型针对多个确诊病人的疾病排序结果进行更新，以获得各状态对疾病排序结果进行更新的更新排序结果，然后根据各更新排序结果确定各状态下多个确诊病人的更新疾病结果。其中，更新疾病结果为更新确诊病人的疾病排序结果后的获得概率最高的疾病。
S042：根据多个确诊病人的疾病诊断结果确定各状态下多个确诊病人的更新疾病结果的准确率，以获得各状态下的疾病结果准确率。
在获得各状态下多个确诊病人的更新疾病结果之后，根据多个确诊病人的疾病诊断结果确定各状态下多个确诊病人的更新疾病结果的准确率，以获得各状态下的疾病结果准确率。
例如，在某状态下，有m个确诊病人的疾病诊断结果，该状态下也对应的有m个确诊 病人的更新疾病结果，其中，m个确诊病人的更新疾病结果中，有n个确诊病人的更新疾病结果与确诊病人的疾病诊断结果一致，则该状态下更新疾病结果的准确率为n/m，重复上述步骤，最终获得不同状态下的更新疾病结果的准确率。
其中，预训练模型的初始状态的k维向量，是各类优势病种的平均准确率，即在初始状态下对辅助诊断模型的多个疾病排序结果进行更新，获得的多个更新疾病结果的平均准确率。
S043：根据前后两个状态下的疾病结果准确率确定预训练模型中下一状态的奖励。
在获得各状态下的疾病结果准确率之后，根据前后两个状态下的疾病结果准确率确定预训练模型中下一状态的奖励。
例如，accu before表示根据上一次状态下的疾病结果准确率，accu now表示当前状态下的疾病结果准确率，限值threshold为0.01，预设训练模型确定奖励的过程为：若|accu before-accu now|>threshold，且accu before<accu now，表示更新后的疾病结果的准确性得到提升，则预设训练模型的奖励值为1；若|accu before-accu now|<threshold，表示更新后的疾病结果的准确性不变，则预设训练模型的奖励值为0；|accu before—accu now|>threshold，且accu before>accu now，表示更新后的疾病结果的准确性下降，则预设训练模型的奖励值为-1。
本实施例中，threshold为0.01仅为示例性说明，在其他实施例中，threshold还可以是其他小于0.01的值，在此不再赘述。
本实施例中，通过根据各状态下优势病种的权重更新辅助诊断模型针对多个确诊病人的疾病排序结果，以确定各状态下多个确诊病人的更新疾病结果，进而根据多个确诊病人的疾病诊断结果确定各状态下多个确诊病人的更新疾病结果的准确率，以获得各状态下的疾病结果准确率，根据前后两个状态下的疾病结果准确率确定预训练模型中下一状态的奖励，细化了根据病人所属地区的疾病诊断数据确定预训练模型的奖励的过程，为奖励的确定提供了基础，使得训练过程中的预训练模型结合辅助诊断模型输出的疾病性能接近确诊病人的实际疾病诊断结果，从而提高了预设权重模型的准确性。
应理解，上述实施例中各步骤的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。
在一实施例中，提供一种基于强化学习模型的疾病排序装置，该基于强化学习模型的疾病排序装置与上述实施例中基于强化学习模型的疾病排序方法一一对应。如图8所示，该基于强化学习模型的疾病排序装置包括第一获取模块801、第二获取模块802、第一确定模块803、更新模块804和第二确定模块805。各功能模块详细说明如下：
第一获取模块801，用于获取病人的病情数据，并将所述病人的病情数据输入辅助诊断模型；
第二获取模块802，用于获取所述辅助诊断模型输出的疾病排序结果，所述疾病排序结果为根据所述病人获得各疾病的概率大小对多个疑似疾病进行排序的结果；
第一确定模块803，用于根据预设权重模型确定所述多个疑似疾病在所述病人所属地 区的权重，所述预设权重模型为根据所述病人所属地区的疾病诊断数据进行疾病权重学习获得的强化学习模型；
更新模块804，用于根据所述多个疑似疾病在所述病人所属地区的权重对所述疑似疾病排序结果进行更新，以获得更新后的疾病排序结果；
第二确定模块805，用于根据所述更新后的疾病排序结果确定所述病人的疑似疾病排序结果，并进行输出。
进一步地，所述第一确定模块803具体用于：
将所述病人所属地区的预设权重模型输出的状态作为所述病人所属地区中多个优势病种的权重；
确定所述多个疑似疾病中各所述疑似疾病的疾病种类；
确定各所述疑似疾病的疾病种类是否为所述病人所属地区的多个优势病种；
若所述疑似疾病的疾病种类为所述病人所属地区的多个优势病种，则将所述优势病种的权重作为对应疑似疾病的权重，以获得所述多个疑似疾病的权重。
进一步地，所述更新模块804具体用于：
根据所述疑似疾病排序结果确定各所述疑似疾病的获得概率；
确定所述疑似疾病在所述病人所属地区的权重与所述疑似疾病的获得概率之间的乘积，以作为所述疑似疾病的最终获得概率；
根据各所述疑似疾病的最终获得概率对多个所述疑似疾病的排序进行更新。
进一步地，所述第二确定模块805具体用于：
在所述更新后的疾病排序结果中确定所述疑似疾病的获得概率；
根据所述疑似疾病的获得概率的大小对所述疑似疾病进行由高到低的排序，获得疑似疾病排序列表；
在所述疑似疾病排序列表中选取前预设数量个所述疑似疾病和所述疑似疾病的获得概率作为所述病人的疑似疾病排序结果。
进一步地，所述基于强化学习模型的疾病排序装置还包括模型训练模块806，所述模型训练模块806具体用于：
确定所述病人所属地区的k个优势病种，所述优势病种为所述病人所属地区中疾病出现频率较高的多个疾病种类；
将k个优势病种的权重定义为预训练模型的状态，所述状态为k维的向量；
将所述k维的向量输入所述预训练模型的神经网络中，以获得所述预训练模型的动作；
根据所述病人所属地区的疾病诊断数据确定所述预训练模型的奖励；
调整所述状态、所述动作和所述奖励以对所述预训练模型进行权重学习，获得所述预设权重模型。
进一步地，所述病人所属地区的疾病诊断数据包括多个确诊病人的疾病诊断结果和所述辅助诊断模型针对多个确诊病人的疾病排序结果，所述模型训练模块806具体还用于：
根据各状态下优势病种的权重更新所述辅助诊断模型针对多个确诊病人的疾病排序结果，以确定各状态下所述多个确诊病人的更新疾病结果，所述更新疾病结果为更新所述确诊病人的疾病排序结果后的获得概率最高的疾病；
根据所述多个确诊病人的疾病诊断结果确定所述各状态下所述多个确诊病人的更新疾病结果的准确率，以获得各状态下的疾病结果准确率；
根据前后两个状态下的疾病结果准确率确定所述预训练模型中下一状态的奖励。
关于基于强化学习模型的疾病排序装置的具体限定可以参见上文中对于基于强化学习模型的疾病排序方法的限定，在此不再赘述。上述基于强化学习模型的疾病排序装置中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各模块可以硬件形式内嵌于或独立于计算机设备中的处理器中，也可以以软件形式存储于计算机设备中的存储器中，以便于处理器调用执行以上各个模块对应的操作。
在一个实施例中，提供了一种计算机设备，该计算机设备可以是服务器，其内部结构图可以如图9所示。该计算机设备包括通过***总线连接的处理器、存储器、网络接口和数据库。其中，该计算机设备的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作***、计算机可读指令和数据库。该内存储器为非易失性存储介质中的操作***和计算机可读指令的运行提供环境。该计算机设备的数据库用于存储辅助诊断模型、预设权重模型和疾病排序结果等。该计算机设备的网络接口用于与外部的终端通过网络连接通信。该计算机可读指令被处理器执行时以实现一种基于强化学习模型的疾病排序方法。
在一个实施例中，提供了一种计算机设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机可读指令，处理器执行计算机可读指令时实现以下步骤：
获取病人的病情数据，并将所述病人的病情数据输入辅助诊断模型；
获取所述辅助诊断模型输出的疾病排序结果，所述疾病排序结果为根据所述病人获得各疾病的概率大小对多个疑似疾病进行排序的结果；
根据预设权重模型确定所述多个疑似疾病在所述病人所属地区的权重，所述预设权重模型为根据所述病人所属地区的疾病诊断数据进行疾病权重学习获得的强化学习模型；
根据所述多个疑似疾病在所述病人所属地区的权重对所述疑似疾病排序结果进行更新，以获得更新后的疾病排序结果；
根据所述更新后的疾病排序结果确定所述病人的疑似疾病排序结果，并进行输出。
在一个实施例中，提供了一种计算机可读存储介质，其上存储有计算机可读指令，计算机可读指令被处理器执行时实现以下步骤：
获取病人的病情数据，并将所述病人的病情数据输入辅助诊断模型；
获取所述辅助诊断模型输出的疾病排序结果，所述疾病排序结果为根据所述病人获得各疾病的概率大小对多个疑似疾病进行排序的结果；
根据预设权重模型确定所述多个疑似疾病在所述病人所属地区的权重，所述预设权重 模型为根据所述病人所属地区的疾病诊断数据进行疾病权重学习获得的强化学习模型；
根据所述多个疑似疾病在所述病人所属地区的权重对所述疑似疾病排序结果进行更新，以获得更新后的疾病排序结果；
根据所述更新后的疾病排序结果确定所述病人的疑似疾病排序结果，并进行输出。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机可读指令来指令相关的硬件来完成，计算机可读指令可存储于计算机可读取存储介质中，所述计算机可读存储介质可以是非易失性，也可以是易失性，该计算机可读指令在执行时，可包括如上述各方法的实施例的流程。其中，本申请所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用，均可包括非易失性和/或易失性存储器。非易失性存储器可包括只读存储器(ROM)、可编程ROM(PROM)、电可编程ROM(EPROM)、电可擦除可编程ROM(EEPROM)或闪存。易失性存储器可包括随机存取存储器(RAM)或者外部高速缓冲存储器。作为说明而非局限，RAM以多种形式可得，诸如静态RAM(SRAM)、动态RAM(DRAM)、同步DRAM(SDRAM)、双数据率SDRAM(DDRSDRAM)、增强型SDRAM(ESDRAM)、同步链路(Synchlink)DRAM(SLDRAM)、存储器总线(Rambus)直接RAM(RDRAM)、直接存储器总线动态RAM(DRDRAM)、以及存储器总线动态RAM(RDRAM)等。
所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，仅以上述各功能单元、模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能单元、模块完成，即将装置的内部结构划分成不同的功能单元或模块，以完成以上描述的全部或者部分功能。
以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围，均应包含在本申请的保护范围之内。

Claims (20)

  1. 一种基于强化学习模型的疾病排序方法，其中，包括：
    获取病人的病情数据，并将所述病人的病情数据输入辅助诊断模型；
    获取所述辅助诊断模型输出的疾病排序结果，所述疾病排序结果为根据所述病人获得各疾病的概率大小对多个疑似疾病进行排序的结果；
    根据预设权重模型确定所述多个疑似疾病在所述病人所属地区的权重，所述预设权重模型为根据所述病人所属地区的疾病诊断数据进行疾病权重学习获得的强化学习模型；
    根据所述多个疑似疾病在所述病人所属地区的权重对所述疑似疾病排序结果进行更新，以获得更新后的疾病排序结果；
    根据所述更新后的疾病排序结果确定所述病人的疑似疾病排序结果，并进行输出。
  2. 如权利要求1所述的基于强化学习模型的疾病排序方法，其中，所述预设权重模型通过如下方式获取：
    确定所述病人所属地区的k个优势病种，所述优势病种为所述病人所属地区中疾病出现频率高于预设频率的多个疾病种类；
    将k个优势病种的权重定义为预训练模型的状态，所述状态为k维的向量；
    将所述k维的向量输入所述预训练模型的神经网络中，以获得所述预训练模型的动作；
    根据所述病人所属地区的疾病诊断数据确定所述预训练模型的奖励；
    调整所述状态、所述动作和所述奖励以对所述预训练模型进行权重学习，获得所述预设权重模型。
  3. 如权利要求2所述的基于强化学习模型的疾病排序方法，其中，所述病人所属地区的疾病诊断数据包括多个确诊病人的疾病诊断结果和所述辅助诊断模型针对多个确诊病人的疾病排序结果，所述根据所述病人所属地区的疾病诊断数据确定所述预训练模型的奖励，包括：
    根据各状态下优势病种的权重更新所述辅助诊断模型针对多个确诊病人的疾病排序结果，以确定各状态下所述多个确诊病人的更新疾病结果，所述更新疾病结果为更新所述确诊病人的疾病排序结果后的获得概率最高的疾病；
    根据所述多个确诊病人的疾病诊断结果确定所述各状态下所述多个确诊病人的更新疾病结果的准确率，以获得各状态下的疾病结果准确率；
    根据前后两个状态下的疾病结果准确率确定所述预训练模型中下一状态的奖励。
  4. 如权利要求1所述的基于强化学习模型的疾病排序方法，其中，所述根据预设权重模型确定所述多个疑似疾病在所述病人所属地区的权重，包括：
    将所述病人所属地区的预设权重模型输出的状态作为所述病人所属地区中多个优势病种的权重；
    确定所述多个疑似疾病中各所述疑似疾病的疾病种类；
    确定各所述疑似疾病的疾病种类是否为所述病人所属地区的多个优势病种；
    若所述疑似疾病的疾病种类为所述病人所属地区的多个优势病种，则将所述优势病种的权重作为对应疑似疾病的权重，以获得所述多个疑似疾病的权重。
  5. 如权利要求1-4任一项所述的基于强化学习模型的疾病排序方法，其中，所述根据所述多个疑似疾病在所述病人所属地区的权重对所述疑似疾病排序结果进行更新，包括：
    根据所述疑似疾病排序结果确定各所述疑似疾病的获得概率；
    确定所述疑似疾病在所述病人所属地区的权重与所述疑似疾病的获得概率之间的乘积，以作为所述疑似疾病的最终获得概率；
    根据各所述疑似疾病的最终获得概率对多个所述疑似疾病的排序进行更新。
  6. 如权利要求1-4任一项所述的基于强化学习模型的疾病排序方法，其中，所述根据所述更新后的疾病排序结果确定所述病人的疑似疾病排序结果，包括：
    在所述更新后的疾病排序结果中确定所述疑似疾病的获得概率；
    根据所述疑似疾病的获得概率的大小对所述疑似疾病进行由高到低的排序，获得疑似疾病排序列表；
    在所述疑似疾病排序列表中选取前预设数量个所述疑似疾病和所述疑似疾病的获得概率作为所述病人的疑似疾病排序结果。
  7. 一种基于强化学习模型的疾病排序装置，其中，包括：
    第一获取模块，用于获取病人的病情数据，并将所述病人的病情数据输入辅助诊断模型；
    第二获取模块，用于获取所述辅助诊断模型输出的疾病排序结果，所述疾病排序结果为根据所述病人获得各疾病的概率大小对多个疑似疾病进行排序的结果；
    第一确定模块，用于根据预设权重模型确定所述多个疑似疾病在所述病人所属地区的权重，所述预设权重模型为根据所述病人所属地区的疾病诊断数据进行疾病权重学习获得的强化学习模型；
    更新模块，用于根据所述多个疑似疾病在所述病人所属地区的权重对所述疑似疾病排序结果进行更新，以获得更新后的疾病排序结果；
    第二确定模块，用于根据所述更新后的疾病排序结果确定所述病人的疑似疾病排序结果，并进行输出。
  8. 如权利要求7所述的基于强化学习模型的疾病排序装置，其中，所述第一确定模块具体用于：
    将所述所属地区的预设权重模型输出的状态作为所述所属地区中多个优势病种的权重；
    确定所述多个疑似疾病中各所述疑似疾病的疾病种类；
    确定各所述疑似疾病的疾病种类是否为所述病人所属地区的多个优势病种；
    若所述疑似疾病的疾病种类为所述病人所属地区的多个优势病种，则将所述优势病种 的权重作为所述疑似疾病的权重，以获得所述多个疾病的权重。
  9. 一种计算机设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机可读指令，其中，所述处理器执行所述计算机可读指令时实现如下步骤：
    获取病人的病情数据，并将所述病人的病情数据输入辅助诊断模型；
    获取所述辅助诊断模型输出的疾病排序结果，所述疾病排序结果为根据所述病人获得各疾病的概率大小对多个疑似疾病进行排序的结果；
    根据预设权重模型确定所述多个疑似疾病在所述病人所属地区的权重，所述预设权重模型为根据所述病人所属地区的疾病诊断数据进行疾病权重学习获得的强化学习模型；
    根据所述多个疑似疾病在所述病人所属地区的权重对所述疑似疾病排序结果进行更新，以获得更新后的疾病排序结果；
    根据所述更新后的疾病排序结果确定所述病人的疑似疾病排序结果，并进行输出。
  10. 如权利要求9所述的计算机设备，其中，所述预设权重模型通过如下方式获取：
    确定所述病人所属地区的k个优势病种，所述优势病种为所述病人所属地区中疾病出现频率高于预设频率的多个疾病种类；
    将k个优势病种的权重定义为预训练模型的状态，所述状态为k维的向量；
    将所述k维的向量输入所述预训练模型的神经网络中，以获得所述预训练模型的动作；
    根据所述病人所属地区的疾病诊断数据确定所述预训练模型的奖励；
    调整所述状态、所述动作和所述奖励以对所述预训练模型进行权重学习，获得所述预设权重模型。
  11. 如权利要求10所述的计算机设备，其中，所述病人所属地区的疾病诊断数据包括多个确诊病人的疾病诊断结果和所述辅助诊断模型针对多个确诊病人的疾病排序结果，所述根据所述病人所属地区的疾病诊断数据确定所述预训练模型的奖励，包括：
    根据各状态下优势病种的权重更新所述辅助诊断模型针对多个确诊病人的疾病排序结果，以确定各状态下所述多个确诊病人的更新疾病结果，所述更新疾病结果为更新所述确诊病人的疾病排序结果后的获得概率最高的疾病；
    根据所述多个确诊病人的疾病诊断结果确定所述各状态下所述多个确诊病人的更新疾病结果的准确率，以获得各状态下的疾病结果准确率；
    根据前后两个状态下的疾病结果准确率确定所述预训练模型中下一状态的奖励。
  12. 如权利要求9所述的计算机设备，其中，所述根据预设权重模型确定所述多个疑似疾病在所述病人所属地区的权重，包括：
    将所述病人所属地区的预设权重模型输出的状态作为所述病人所属地区中多个优势病种的权重；
    确定所述多个疑似疾病中各所述疑似疾病的疾病种类；
    确定各所述疑似疾病的疾病种类是否为所述病人所属地区的多个优势病种；
    若所述疑似疾病的疾病种类为所述病人所属地区的多个优势病种，则将所述优势病种 的权重作为对应疑似疾病的权重，以获得所述多个疑似疾病的权重。
  13. 如权利要求9-12任一项所述的计算机设备，其中，所述根据所述多个疑似疾病在所述病人所属地区的权重对所述疑似疾病排序结果进行更新，包括：
    根据所述疑似疾病排序结果确定各所述疑似疾病的获得概率；
    确定所述疑似疾病在所述病人所属地区的权重与所述疑似疾病的获得概率之间的乘积，以作为所述疑似疾病的最终获得概率；
    根据各所述疑似疾病的最终获得概率对多个所述疑似疾病的排序进行更新。
  14. 如权利要求9-12任一项所述的计算机设备，其中，所述根据所述更新后的疾病排序结果确定所述病人的疑似疾病排序结果，包括：
    在所述更新后的疾病排序结果中确定所述疑似疾病的获得概率；
    根据所述疑似疾病的获得概率的大小对所述疑似疾病进行由高到低的排序，获得疑似疾病排序列表；
    在所述疑似疾病排序列表中选取前预设数量个所述疑似疾病和所述疑似疾病的获得概率作为所述病人的疑似疾病排序结果。
  15. 一个或多个存储有计算机可读指令的可读存储介质，其中，所述计算机可读指令被一个或多个处理器执行时，使得所述一个或多个处理器执行如下步骤：
    获取病人的病情数据，并将所述病人的病情数据输入辅助诊断模型；
    获取所述辅助诊断模型输出的疾病排序结果，所述疾病排序结果为根据所述病人获得各疾病的概率大小对多个疑似疾病进行排序的结果；
    根据预设权重模型确定所述多个疑似疾病在所述病人所属地区的权重，所述预设权重模型为根据所述病人所属地区的疾病诊断数据进行疾病权重学习获得的强化学习模型；
    根据所述多个疑似疾病在所述病人所属地区的权重对所述疑似疾病排序结果进行更新，以获得更新后的疾病排序结果；
    根据所述更新后的疾病排序结果确定所述病人的疑似疾病排序结果，并进行输出。
  16. 如权利要求15所述的可读存储介质，其中，所述预设权重模型通过如下方式获取：
    确定所述病人所属地区的k个优势病种，所述优势病种为所述病人所属地区中疾病出现频率高于预设频率的多个疾病种类；
    将k个优势病种的权重定义为预训练模型的状态，所述状态为k维的向量；
    将所述k维的向量输入所述预训练模型的神经网络中，以获得所述预训练模型的动作；
    根据所述病人所属地区的疾病诊断数据确定所述预训练模型的奖励；
    调整所述状态、所述动作和所述奖励以对所述预训练模型进行权重学习，获得所述预设权重模型。
  17. 如权利要求16所述的可读存储介质，其中，所述病人所属地区的疾病诊断数据包括多个确诊病人的疾病诊断结果和所述辅助诊断模型针对多个确诊病人的疾病排序结果，所述根据所述病人所属地区的疾病诊断数据确定所述预训练模型的奖励，包括：
    根据各状态下优势病种的权重更新所述辅助诊断模型针对多个确诊病人的疾病排序结果，以确定各状态下所述多个确诊病人的更新疾病结果，所述更新疾病结果为更新所述确诊病人的疾病排序结果后的获得概率最高的疾病；
    根据所述多个确诊病人的疾病诊断结果确定所述各状态下所述多个确诊病人的更新疾病结果的准确率，以获得各状态下的疾病结果准确率；
    根据前后两个状态下的疾病结果准确率确定所述预训练模型中下一状态的奖励。
  18. 如权利要求15所述的可读存储介质，其中，所述根据预设权重模型确定所述多个疑似疾病在所述病人所属地区的权重，包括：
    将所述病人所属地区的预设权重模型输出的状态作为所述病人所属地区中多个优势病种的权重；
    确定所述多个疑似疾病中各所述疑似疾病的疾病种类；
    确定各所述疑似疾病的疾病种类是否为所述病人所属地区的多个优势病种；
    若所述疑似疾病的疾病种类为所述病人所属地区的多个优势病种，则将所述优势病种的权重作为对应疑似疾病的权重，以获得所述多个疑似疾病的权重。
  19. 如权利要求15-18任一项所述的可读存储介质，其中，所述根据所述多个疑似疾病在所述病人所属地区的权重对所述疑似疾病排序结果进行更新，包括：
    根据所述疑似疾病排序结果确定各所述疑似疾病的获得概率；
    确定所述疑似疾病在所述病人所属地区的权重与所述疑似疾病的获得概率之间的乘积，以作为所述疑似疾病的最终获得概率；
    根据各所述疑似疾病的最终获得概率对多个所述疑似疾病的排序进行更新。
  20. 如权利要求15-18任一项所述的可读存储介质，其中，所述根据所述更新后的疾病排序结果确定所述病人的疑似疾病排序结果，包括：
    在所述更新后的疾病排序结果中确定所述疑似疾病的获得概率；
    根据所述疑似疾病的获得概率的大小对所述疑似疾病进行由高到低的排序，获得疑似疾病排序列表；
    在所述疑似疾病排序列表中选取前预设数量个所述疑似疾病和所述疑似疾病的获得概率作为所述病人的疑似疾病排序结果。
PCT/CN2020/135340 2020-09-07 2020-12-10 基于强化学习模型的疾病排序方法、装置、设备及介质 WO2021151355A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010929683.4A CN112017788B (zh) 2020-09-07 2020-09-07 基于强化学习模型的疾病排序方法、装置、设备及介质
CN202010929683.4 2020-09-07

Publications (1)

Publication Number Publication Date
WO2021151355A1 true WO2021151355A1 (zh) 2021-08-05

Family

ID=73517050

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/135340 WO2021151355A1 (zh) 2020-09-07 2020-12-10 基于强化学习模型的疾病排序方法、装置、设备及介质

Country Status (2)

Country Link
CN (1) CN112017788B (zh)
WO (1) WO2021151355A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112017788B (zh) * 2020-09-07 2023-07-04 平安科技（深圳）有限公司 基于强化学习模型的疾病排序方法、装置、设备及介质
CN113284623B (zh) * 2021-07-23 2021-11-05 北京智精灵科技有限公司 基于用户能力的个性化认知训练任务推荐算法及***

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160120481A1 (en) * 2014-10-30 2016-05-05 International Business Machines Corporation Active patient risk prediction
CN109102869A (zh) * 2018-08-22 2018-12-28 泰康保险集团股份有限公司 基于区块链的医疗数据管理方法、装置、介质及电子设备
CN109147931A (zh) * 2017-06-16 2019-01-04 宏达国际电子股份有限公司 电脑辅助医疗法、非暂态电脑可读取存储媒体及医疗***
CN109671506A (zh) * 2017-10-16 2019-04-23 南京唯实科技有限公司 一种基于大数据的疾病预测分析方法
CN109891517A (zh) * 2016-10-25 2019-06-14 皇家飞利浦有限公司 基于知识图的临床诊断助手
CN110993103A (zh) * 2019-11-28 2020-04-10 阳光人寿保险股份有限公司 疾病风险预测模型的建立方法和疾病保险产品的推荐方法
CN112017788A (zh) * 2020-09-07 2020-12-01 平安科技（深圳）有限公司 基于强化学习模型的疾病排序方法、装置、设备及介质

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110599476B (zh) * 2019-09-12 2023-05-23 腾讯科技（深圳）有限公司 基于机器学习的疾病分级方法、装置、设备及介质
CN111599427B (zh) * 2020-05-14 2023-03-31 郑州大学第一附属医院 一种一元化诊断的推荐方法、装置、电子设备及存储介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160120481A1 (en) * 2014-10-30 2016-05-05 International Business Machines Corporation Active patient risk prediction
CN109891517A (zh) * 2016-10-25 2019-06-14 皇家飞利浦有限公司 基于知识图的临床诊断助手
CN109147931A (zh) * 2017-06-16 2019-01-04 宏达国际电子股份有限公司 电脑辅助医疗法、非暂态电脑可读取存储媒体及医疗***
CN109671506A (zh) * 2017-10-16 2019-04-23 南京唯实科技有限公司 一种基于大数据的疾病预测分析方法
CN109102869A (zh) * 2018-08-22 2018-12-28 泰康保险集团股份有限公司 基于区块链的医疗数据管理方法、装置、介质及电子设备
CN110993103A (zh) * 2019-11-28 2020-04-10 阳光人寿保险股份有限公司 疾病风险预测模型的建立方法和疾病保险产品的推荐方法
CN112017788A (zh) * 2020-09-07 2020-12-01 平安科技（深圳）有限公司 基于强化学习模型的疾病排序方法、装置、设备及介质

Also Published As

Publication number Publication date
CN112017788A (zh) 2020-12-01
CN112017788B (zh) 2023-07-04

Similar Documents

Publication Publication Date Title
CN112037912B (zh) 基于医疗知识图谱的分诊模型训练方法、装置及设备
WO2021121126A1 (zh) 基于深度学习的脑瘤图像分割方法、装置、设备及介质
Hu et al. Automatic construction of Chinese herbal prescriptions from tongue images using CNNs and auxiliary latent therapy topics
KR102424085B1 (ko) 기계-보조 대화 시스템 및 의학적 상태 문의 장치 및 방법
WO2021151355A1 (zh) 基于强化学习模型的疾病排序方法、装置、设备及介质
US20220326971A1 (en) User interface for recommending insertion of an updated medical best practice recommendation in response to user entry of updated medical observation informaton for a patient
WO2021114620A1 (zh) 病历质控方法、装置、计算机设备和存储介质
WO2021057884A1 (zh) 语句复述方法、训练语句复述模型的方法及其装置
WO2020172607A1 (en) Systems and methods for using deep learning to generate acuity scores for critically ill or injured patients
CN113724814B (zh) 一种分诊方法、装置、计算设备及存储介质
WO2021114635A1 (zh) 患者分群模型构建方法、患者分群方法及相关设备
CN110808095B (zh) 诊断结果识别、模型训练的方法、计算机设备及存储介质
WO2021151322A1 (zh) 基于深度学习模型的实体识别方法、装置、设备及介质
CN114628001B (zh) 基于神经网络的处方推荐方法、***、设备及存储介质
Yang et al. NAUNet: lightweight retinal vessel segmentation network with nested connections and efficient attention
WO2022227171A1 (zh) 关键信息提取方法、装置、电子设备及介质
WO2021174923A1 (zh) 概念词序列生成方法、装置、计算机设备及存储介质
WO2021139223A1 (zh) 分群模型的解释方法、装置、计算机设备和存储介质
Schlosser et al. Visual acuity prediction on real-life patient data using a machine learning based multistage system
Li et al. A transfer learning-based multimodal neural network combining metadata and multiple medical images for glaucoma type diagnosis
WO2023124837A1 (zh) 问诊处理方法、装置、设备及存储介质
CN108319580A (zh) 诊断词归一方法及装置
CN116719840A (zh) 一种基于病历后结构化处理的医疗信息推送方法
CN113672701B (zh) 基于机器学习的文本匹配方法、装置、设备及存储介质
CN115662562A (zh) 病历诊疗数据管理方法、装置、设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20917194

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20917194

Country of ref document: EP

Kind code of ref document: A1