CN113241198B - User data processing method, device, equipment and storage medium - Google Patents
User data processing method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN113241198B CN113241198B CN202110605578.XA CN202110605578A CN113241198B CN 113241198 B CN113241198 B CN 113241198B CN 202110605578 A CN202110605578 A CN 202110605578A CN 113241198 B CN113241198 B CN 113241198B
- Authority
- CN
- China
- Prior art keywords
- data
- image
- mapping
- feature
- probability
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H80/00—ICT specially adapted for facilitating communication between medical practitioners or patients, e.g. for collaborative diagnosis, therapy or health monitoring
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Public Health (AREA)
- Primary Health Care (AREA)
- Biomedical Technology (AREA)
- Physics & Mathematics (AREA)
- Epidemiology (AREA)
- Theoretical Computer Science (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Data Mining & Analysis (AREA)
- Pathology (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Medical Treatment And Welfare Office Work (AREA)
Abstract
The present invention relates to the field of information technologies, and in particular, to a user data processing method, a user data processing apparatus, a computer device, and a storage medium. The method comprises the following steps: acquiring user data of a patient, wherein the user data comprises image data, text data, audio data and structured data; acquiring a pre-trained decoder corresponding to the image data, the text data and the audio data; decoding the image data, the text data and the audio data based on the decoder respectively to obtain corresponding data characteristics; mapping each data characteristic and the structured data to a probability space to obtain a characteristic mapping image; according to each data feature and the feature mapping image, the disease type and the corresponding probability are determined, so that the intelligent inquiry is realized, and the inquiry efficiency is improved.
Description
Technical Field
The present invention relates to the field of information technologies, and in particular, to a user data processing method, a user data processing device, a computer device, and a storage medium.
Background
The existing intelligent inquiry system is mainly designed from the point of view of convenience of patients, and is mainly characterized in that the flow is facilitated, such as self-diagnosis, sub-diagnosis and other services are provided before registration of the patients; after successful registration, before the doctor receives the diagnosis, the patient informs the state of the illness in advance through pre-consultation, and the doctor collects the information of the patient in advance, so that the optimization and configuration of the diagnosis and treatment flow are realized. However, the existing intelligent inquiry system only reflects the convenience of acquiring the disease information of the patient, but does not provide technical support or advice for the patient and doctor by utilizing the disease information, the doctor still needs to spend a great deal of effort to analyze the disease of the patient, a great deal of time is wasted, and the inquiry efficiency is low.
Disclosure of Invention
The application provides a user data processing method, a user data processing device, computer equipment and a storage medium, and aims to solve the problem of low inquiry efficiency of an existing inquiry system.
To achieve the above object, the present application provides a user data processing method, including:
acquiring user data of a patient, wherein the user data comprises image data, text data, audio data and structured data;
acquiring a pre-trained decoder corresponding to the image data, the text data and the audio data;
decoding the image data, the text data and the audio data based on the decoder respectively to obtain corresponding data characteristics;
mapping each data characteristic and each structured data to a probability space to obtain a characteristic mapping image;
and determining the disease type and the corresponding probability according to each data characteristic and the characteristic mapping image.
To achieve the above object, the present application further provides a user data processing apparatus, including:
the data acquisition module is used for acquiring user data of a patient, wherein the user data comprises image data, text data, audio data and structured data;
the decoder acquisition module is used for acquiring the pre-trained decoder corresponding to the image data, the text data and the audio data;
the feature extraction module is used for respectively decoding the image data, the text data and the audio data based on the decoder to obtain corresponding data features;
the feature mapping module is used for mapping each data feature and the structured data to a probability space to obtain a feature mapping image;
and the data processing module is used for determining the disease type and the corresponding probability according to each data characteristic and the characteristic mapping image.
In addition, to achieve the above object, the present application further provides a computer apparatus including a memory and a processor; the memory is used for storing a computer program; the processor is configured to execute the computer program and implement any one of the user data processing methods provided in the embodiments of the present application when the computer program is executed.
In addition, to achieve the above object, the present application further provides a computer readable storage medium storing a computer program, where the computer program when executed by a processor causes the processor to implement the user data processing method according to any one of the embodiments provided herein.
According to the user data processing method, the user data processing device, the equipment and the storage medium disclosed by the embodiment of the application, the intellectualization of the inquiry system is realized by analyzing the illness state information, doctors can be assisted in carrying out illness state consultation and illness state judgment, inquiry efficiency is improved, and user experience is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is an application scenario diagram of a user data processing method provided in an embodiment of the present application;
fig. 2 is a flow chart of a user data processing method according to an embodiment of the present application;
FIG. 3 is a schematic flow chart of mapping user data to probability space according to an embodiment of the present application;
FIG. 4 is a flow chart illustrating the generation of feature map images from user data provided by embodiments of the present application;
FIG. 5 is a schematic block diagram of a user data processing apparatus provided in an embodiment of the present application;
fig. 6 is a schematic block diagram of a computer device provided in an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made with reference to the accompanying drawings, in which it is apparent that some, but not all embodiments of the embodiments described are described. All other embodiments, based on the embodiments herein, which would be apparent to one of ordinary skill in the art without making any inventive effort, are intended to be within the scope of the present application.
The flow diagrams depicted in the figures are merely illustrative and not necessarily all of the elements and operations/steps are included or performed in the order described. For example, some operations/steps may be further divided, combined, or partially combined, so that the order of actual execution may be changed according to actual situations. In addition, although the division of the functional modules is performed in the apparatus schematic diagram, in some cases, the division of the modules may be different from that in the apparatus schematic diagram.
The term "and/or" as used in this specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
The existing intelligent inquiry system is mainly used for collecting user data such as CT images, such as patient self-description symptoms and the like in advance, so that the consultation time of doctors and patients for physical conditions can be saved, optimization and configuration of an diagnosis process are realized, and inquiry efficiency is improved. However, the existing intelligent inquiry system only shows convenience for acquiring the disease information of the patient, but does not provide technical support or advice for the patient and doctor by utilizing the disease information, the doctor still needs to spend a great deal of effort to analyze the disease of the patient, a great deal of time is wasted, and the inquiry efficiency is low.
Based on the problems, the application provides a user data processing method for solving the problem of low inquiry efficiency of the existing inquiry system, so that the intelligent inquiry is realized, the disease prediction result can be provided, the doctor is assisted in carrying out disease consultation and disease judgment, and the inquiry efficiency is improved.
The user data processing method can be applied to a server or an intelligent inquiry system, so that the inquiry intellectualization is realized, wherein the server can be a single server or a server cluster. For ease of understanding, however, the following embodiments will be described in detail with respect to a user data processing method applied to a server.
Some embodiments of the present application are described in detail below with reference to the accompanying drawings. The following embodiments and features of the embodiments may be combined with each other without conflict.
Referring to fig. 1, fig. 1 is an application scenario diagram of a user data processing method provided in an embodiment of the present application, where, as shown in fig. 1, the user data processing method provided in the embodiment of the present application may be applied to an application environment shown in fig. 1. The application environment includes a server 110, a terminal device 120 and a hospital information system 130, wherein the server 110 can communicate with the terminal device 120 and the hospital information system 130 through a network. Specifically, the terminal device 120 may send text data, audio data, structured data, etc. to the server 110, and the hospital information system 130 may also be communicatively connected to the medical device for acquiring image data, such as CT images, etc., so that the hospital information system 130 may send the image data, text data, structured data, etc. to the server 110, and finally the server 110 determines the kind of illness and the corresponding probability based on the above user data. The server 120 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, basic cloud computing services such as big data and an artificial intelligence platform, and specifically, the server 110 may also be an intelligent inquiry system including a server. The terminal device 110 may be, but is not limited to, a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, etc. The terminal device, the server, and the hospital information system may be directly or indirectly connected through a wired or wireless communication manner, which is not limited herein.
Referring to fig. 2, fig. 2 is a schematic flowchart of a user data processing method according to an embodiment of the present application. The user data processing method can be applied to a server or an intelligent consultation platform, so that the intelligent consultation is realized, doctors can be assisted in carrying out condition consultation and disease judgment, the consultation efficiency is improved, and the user experience is improved.
As shown in fig. 2, the user data processing method includes steps S101 to S105.
S101, acquiring user data of a patient, wherein the user data comprises image data, text data, audio data and structured data.
In particular, user data of the patient is acquired, including but not limited to image data, text data, audio data, and structured data. The image data are medical detection images such as CT images, B-ultrasonic images, nuclear magnetic resonance images and the like obtained through medical equipment; the text data is pre-input illness state information of a patient such as bellyache, headache and the like or illness state diagnosis information of a doctor for the patient; the audio data is the sound information of the patient, such as cough, sneeze and the like of the patient; the structured data is the medical detection result of the patient, such as blood routine, height, weight and other information.
For example, when the user feels uncomfortable, the type of possible illness and the corresponding probability can be determined in a simulation mode in the intelligent inquiry system. Specifically, the intelligent inquiry system can acquire image information of a corresponding user through the hospital information system, and the intelligent inquiry system is in communication connection with medical equipment such as a CT machine, so that image data such as a CT image and the like can be acquired at any time.
The intelligent inquiry system is also in communication connection with a terminal device of a user, and the user can input information of illness states such as bellyache, headache and the like on the terminal device, or a doctor can input information of illness states diagnosis of a corresponding patient such as possible diseases or symptoms determined by the doctor according to the description of the user on the terminal device.
For example, the intelligent interrogation system may also record audio information of the user, such as recording the user's own coughs, sneezes, etc. Meanwhile, the user can also input corresponding structured data such as height, weight and other information through an application program (APP) of the terminal equipment, and the terminal equipment sends the user data to the server for processing.
It should be noted that the intelligent inquiry system is also in communication connection with a hospital information system, from which structured data of the patient, such as blood routine, hemoglobin content, etc., can be obtained.
S102, acquiring the pre-trained decoder corresponding to the image data, the text data and the audio data.
Specifically, corresponding pre-trained decoders are determined and acquired according to the type of the user data. Therefore, the corresponding decoder can be determined according to the data type, the decoding speed can be increased, various data can be decoded at the same time, and the decoding efficiency is improved.
The server may first analyze the type of user data, for example, the type of user data to obtain the type of user data as image data, and determine a pre-trained decoder corresponding to the image data, so that the image data may be input to the corresponding decoder for decoding. Similarly, based on the data types obtained by analysis, pre-trained decoders corresponding to the text data and the audio data are respectively determined.
It should be noted that, the structured data is information such as height, weight, blood convention, hemoglobin content, and the like, and since the structured data can be directly mapped into the probability space, decoding processing is not required, and thus a corresponding decoder is not required.
S103, decoding the image data, the text data and the audio data based on the decoder respectively to obtain corresponding data characteristics.
And respectively decoding the image data, the text data and the audio data through the decoder to obtain data characteristics corresponding to the image data, the text data and the audio data. Therefore, the corresponding data can be decoded by the corresponding decoder, so that the decoding speed can be increased, various data can be simultaneously decoded, and the decoding efficiency is improved.
Specifically, the image data, the text data and the audio data are decoded by different types of decoders, and the different decoders can obtain the data characteristics corresponding to the image data, the text data and the audio data respectively.
As shown in fig. 3, fig. 3 is a schematic flow chart of mapping user data to a probability space, where a decoder corresponding to image data is determined to be a res net decoder according to a type of the user data, a decoder corresponding to text data is a GloVe decoder, a decoder corresponding to audio data is an audio decoder, the image data is decoded by the res net decoder, the text data is decoded by the GloVe decoder, the audio data is decoded by the audio decoder, and corresponding image features, text features and audio features are respectively obtained, where structured data is directly mapped to the probability space.
In some embodiments, image data is decoded based on a ResNet decoder to obtain image features corresponding to the image data. The ResNet decoder comprises a ResNet50 network structure and a linear layer, wherein the ResNet50 network structure belongs to a Resnet classification network, and is the CNN feature extraction network which is most widely applied at present.
Specifically, the ResNet decoder extracts image features through the ResNet50 network structure and maps the image features into the sample tag space in the form of distributed features by interfacing a linear layer with the ResNet50 network structure. The sample marking space is the space for storing all data features.
Illustratively, a CT image is input into a res net decoder, decoded and analyzed by a res net classification network, corresponding image features such as image shadow features are extracted, and represented in the form of distributed features by a linear layer and mapped into a sample label space.
In some embodiments, text data is decoded based on a GloVe decoder to obtain text features corresponding to the text data. The GloVe decoder includes a pre-trained GloVe model, glove (Global vectors for word representation) is a global word frequency based word characterization tool that can express a word as a vector of real numbers that can capture some semantic characteristics, such as similarity, analogies, etc., between words. The semantic similarity between two words can thus be calculated by performing operations on the vectors, such as calculating euclidean distance or cosine similarity. Based on the method, the GloVe model can decode text information at the word level well, and corresponding text features are output through a GloVe decoder.
Specifically, a GloVe decoder decodes and analyzes text data through a GloVe model, acquires word vectors of each word in the text data by using the GloVe decoder, and determines corresponding text features through a statistical co-occurrence matrix.
Illustratively, text information dictated by the patient, such as the expression of pulmonary pain, is input to a ResNet decoder, which is analyzed by a GloVe model to extract corresponding text features.
In some embodiments, audio data is decoded based on an audio decoder, and audio features corresponding to the audio data are obtained. Wherein the audio decoder is based on a decoder in a Tacotron2 architecture.
Specifically, a Tacotron2 feature prediction network is utilized, the feature prediction network is a cyclic sequence-to-sequence feature prediction network, and features are superimposed on a Mel spectrogram, so that audio features are obtained.
Illustratively, a patient's cough sound is input to an audio decoder, which is decoded and analyzed by a predictive network of sound spectra in the audio decoder to extract corresponding audio features.
It should be noted that, since the structured data may be regarded as a decoded data feature, the structured data may be mapped directly into the probability space without performing decoding processing.
And S104, mapping each data characteristic and the structured data to a probability space to obtain a characteristic mapping image.
Specifically, each data feature obtained through decoding by a decoder and the structured data are mapped to a probability space to obtain a feature mapping image. Wherein the probability space is a measure space with a total measure of 1, and the feature mapping image is a mapping image of one or more data features in the same probability space.
In some embodiments, each of the data features is mapped to a probability space based on a probability cross-Modal Embedding model (PCME), probabilistic Cross-Modal Embedding, resulting in a feature mapped image. The probability cross-modal embedding model is an effective probability representation tool, and the probability cross-modal embedding model can represent the relationship of a plurality of data features in one probability space without representing the data features by using a plurality of probability spaces.
In some embodiments, the data features corresponding to the user data are mapped to a probability space in a single-mode mapping manner, so as to obtain a feature mapping image. Wherein the unimodal mapping method trains unimodal probability embedding by minimizing soft contrast loss.
Specifically, the PCME model treats each data feature as one distribution. It is based on Hedged Instance Embeddings (HIB), a single mode method for representing instances as distributions. Where HIB is a probabilistic simulation of the loss that not only preserves the similarity of the pair-wise semantics, but also represents the uncertainty inherent in the data. While key components of HIB include: soft contrast loss, decomposition matching probability, and euclidean distance based matching probability.
The soft contrast loss is a soft version contrast loss formulated by the HIB, and is widely used for training depth measurement embedding. For a pair of data features (χ α ,χ β ) The soft contrast loss is defined as:
wherein p is θ (m|χ α ，χ β ) To resolve the match probability, a degree of match for a pair of data features can be determined by resolving the match probability when the pair of data features (χ α ，χ β ) In matching, use-lovp θ (m|χ α ，χ β ) This formula calculates the loss when the data characteristic (χ α ，χ β ) When not matched, use-log (1-p θ (m|χ α ，χ β ) This equation calculates the loss. The decomposition matching probability is obtained through a Monte Carlo simulation method, the decomposition matching probability is mapped to Euclidean space to obtain the matching probability based on Euclidean distance, and the single-mode probability embedding is trained by minimizing soft contrast loss.
In some embodiments, each data feature may also be mapped to a probability space by a multi-modal mapping manner, so as to obtain a feature mapping image.
Specifically, the multi-modal mapping method is implemented based on a single-modal mapping method, each data feature is input into a common network structure, the network structure comprises a full-connection layer and a position attention layer, and finally the multi-modal probability embedding is trained by using soft contrast loss minimization in the HIB.
The full connection layer can enable each data feature to be represented in a distributed feature mode and mapped into a sample mark space, different data feature maps are different in positions in the feature map image, and the position attention layer can capture space dependence between any two positions in the feature map image, and for a specific feature, the spatial dependence is weighted and updated by features in all positions. And the weight is the characteristic similarity between the two corresponding positions. Thus, the location of any two existing similar features can be mutually contributed up regardless of the distance between them. In this way, a plurality of types of user data can be mapped into a common probability space, and a feature map image can be obtained.
As shown in fig. 4, fig. 4 is a schematic flow chart of generating a feature map image by using user data, inputting CT images of image data such as chest to a res net decoder for decoding analysis to obtain image features corresponding to the chest CT images, inputting text data such as chest pain to a Govle decoder for decoding analysis to obtain corresponding text features, and mapping the image features and the text features to a common probability space to obtain corresponding feature map images.
In some embodiments, each data feature is filtered according to the feature mapping image, and the filtered data features are remapped to a probability space to obtain a new feature mapping image. Therefore, irrelevant data features can be screened out, and the accuracy of the feature mapping image is improved.
Specifically, after the feature mapping image is obtained, the feature mapping image is analyzed, each data feature is detected on the feature mapping image, irrelevant data features are determined, screening is carried out on the irrelevant data features, and after screening is completed, each data feature after screening is mapped to a probability space again, so that a new feature mapping image is obtained.
For example, if the feature map image is obtained, the feature map image is analyzed, and the other features are found to be related to the lung features, but a text feature is found to be foot pain, at this time, the text feature is used as an irrelevant data feature and filtered, and each filtered data feature is remapped to a probability space, so as to obtain a new feature map image.
It should be noted that, after the irrelevant data features are filtered, corresponding user data may be obtained again for supplementation.
S105, determining the disease type and the corresponding probability according to each data feature and the feature mapping image.
Wherein the disease types can comprise cold, cancer and other diseases, and the probability is the probability corresponding to the cold, cancer and other diseases. Therefore, based on the predicted disease type and the corresponding probability, the doctor can be assisted in disease consultation and judgment, so that the doctor can be assisted in accurately determining the disease condition of the patient.
For example, if the predicted disease type is fever and the corresponding probability is 80%, the doctor may further ask the patient whether symptoms such as dizziness and inappetence related to fever exist based on the above conclusion, and may also perform targeted physical detection on the patient, so as to accurately determine the patient's condition, and greatly improve the inquiry efficiency.
In some embodiments, a disease type is determined from the each data feature and a probability corresponding to the disease type is determined from the feature map image. Specifically, the disease type is determined from one or more of image features, text features, audio features, and structured data.
For example, if a high density shadow is found in the image of the lung, and the patient orally expresses the pain of the lung, the patient can judge that the lung of the patient is possibly problematic from the cough sound of the patient, and then, in combination with some physical indexes of the patient, such as blood detection, the carcinoembryonic antigen is found to be very high, the disease type of the patient can be determined to include cancer, pulmonary tuberculosis, lung inflammation, benign tumor diseases of the lung and the like.
Specifically, the probability corresponding to each disease category may be determined according to the overlapping area of the feature map image by determining the overlapping area of the feature map image. The probability of the patient suffering from the disease can thus be visually seen from the image.
In some embodiments, the disease category and the corresponding probability are determined from the each data feature and the feature map image based on a pre-trained disease classification model. Wherein the disease classification model comprises a fully connected layer and a softmax layer.
Specifically, each data feature and the feature mapping image are input into a pre-trained disease classification model, and finally the disease classification model can output the disease type and the corresponding probability of the patient.
The full connection layer multiplies the weight matrix by the input vector and adds bias to map n disease types into k real numbers, and meanwhile, the softmax layer maps k real numbers into k real numbers (probabilities) in (0, 1) and ensures that the sum of the k real numbers is 1. Illustratively, the probability of cancer, tuberculosis, lung inflammation, and benign tumor disease of the lung were found to be 50%, 30%, and 20%, respectively, at the end.
The specific probability calculation formula is as follows:
γ＝softmax(z)＝softmax(W T x+b)
wherein x is the input of the full connection layer, and specifically each data feature and feature mapping image can be used as the input of the full connection layer, W is the weight, and W T x is the inner product of the weight and the input of the full connection layer, b is the bias term, and gamma is the probability of Softmax output, so that the probability corresponding to each disease type can be calculated. Obtaining the scores of K disease types in the (- + -infinity, + -infinity) range through the full connection layer, and obtaining the probability corresponding to each disease type by firstly using the scoresMapping to (0, + -infinity), and normalizing to (0, 1) to obtain the corresponding probability, the specific softmax calculation formula is:
wherein z is j Can be expressed as a corresponding score for the jth diseased class, in particular, z j ＝w j *x+b j Wherein b j Bias term corresponding to jth disease category, w j For the feature weight corresponding to the jth disease category, namely the importance degree of each feature and the influence degree on the final score, the score of each category is obtained by weighting and summing the features, and then mapped into a rough map through a Softmax functionThe rate.
In some embodiments, after determining the disease type and the corresponding probability, a hint of the predicted completion may also be output.
The prompting information mode can specifically include application program (APP) or means such as Email, short message, chat tool, e.g. WeChat, qq and the like.
Illustratively, after determining the disease type and the corresponding probability, the Application (APP) may transmit a prompt message that the popup alerts the user that the disease prediction has been completed, and the user may also look at the predicted disease type and the corresponding probability on the Application (APP).
It can be appreciated that the user can set the reminding mode by himself, for example, the reminding mode can be set as an Application (APP) and a WeChat reminding mode, and then the reminding information can be sent to the user through the two reminding modes.
Referring to fig. 5, fig. 5 is a schematic block diagram of a user data processing apparatus according to an embodiment of the present application, where the user data processing apparatus may be configured in a server to perform the foregoing user data processing method.
As shown in fig. 5, the user data processing apparatus 200 includes: a data acquisition module 201, a decoder acquisition module 202, a feature extraction module 203, a feature mapping module 204, and a data processing module 205.
A data acquisition module 201 for acquiring user data of a patient, the user data including image data, text data, audio data, and structured data;
a decoder obtaining module 202, configured to obtain a pre-trained decoder corresponding to the image data, the text data, and the audio data;
the feature extraction module 203 decodes the image data, the text data and the audio data based on the decoder, respectively, to obtain corresponding data features;
a feature mapping module 204, configured to map each of the data features and the structured data to a probability space, to obtain a feature mapping image;
the data processing module 205 is configured to determine a disease type and a corresponding probability according to the each data feature and the feature mapping image.
It should be noted that, for convenience and brevity of description, specific working processes of the above-described apparatus and each module, unit may refer to corresponding processes in the foregoing method embodiments, and are not repeated herein.
The methods and apparatus of the present application are operational with numerous general purpose or special purpose computing system environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.
The above-described methods, apparatus may be implemented, for example, in the form of a computer program that is executable on a computer device as shown in fig. 6.
Referring to fig. 6, fig. 6 is a schematic diagram of a computer device according to an embodiment of the present application. The computer device may be a server.
As shown in fig. 6, the computer device includes a processor, a memory, and a network interface connected by a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.
The non-volatile storage medium may store an operating system and a computer program. The computer program comprises program instructions which, when executed, cause the processor to perform any one of a number of user data processing methods.
The processor is used to provide computing and control capabilities to support the operation of the entire computer device.
The internal memory provides an environment for the execution of a computer program in a non-volatile storage medium that, when executed by a processor, causes the processor to perform any one of a number of user data processing methods.
The network interface is used for network communication such as transmitting assigned tasks and the like. It will be appreciated by those skilled in the art that the structure of the computer device is merely a block diagram of some of the structures associated with the present application and is not limiting of the computer device to which the present application may be applied, and that a particular computer device may include more or less components than those shown, or may combine some of the components, or have a different arrangement of components.
It should be appreciated that the processor may be a central processing unit (Central Processing Unit, CPU), but may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. Wherein the general purpose processor may be a micro processor or the processor may be any conventional processor or the like.
Wherein in some embodiments the processor is configured to run a computer program stored in the memory to implement the steps of: acquiring user data of a patient, wherein the user data comprises image data, text data, audio data and structured data; acquiring a pre-trained decoder corresponding to the image data, the text data and the audio data; decoding the image data, the text data and the audio data based on the decoder respectively to obtain corresponding data characteristics; mapping each data characteristic and the structured data to a probability space to obtain a characteristic mapping image; and determining the disease type and the corresponding probability according to each data characteristic and the characteristic mapping image.
In some embodiments, the processor is further configured to: decoding image data based on a ResNet decoder to obtain image features corresponding to the image data; decoding text data based on a GloVe decoder to obtain text features corresponding to the text data; decoding the audio data based on the audio decoder to obtain the audio characteristics corresponding to the audio data.
In some embodiments, the processor is further configured to: and mapping each data feature and the structured data to a probability space based on a probability cross-modal embedding model to obtain a feature mapping image.
In some embodiments, the processor is further configured to: mapping the data features corresponding to the user data to a probability space in a single-mode mapping mode to obtain a feature mapping image; or mapping the data features corresponding to the user data to a probability space in a multi-mode mapping mode to obtain a feature mapping image.
In some embodiments, the processor is further configured to: and determining the diseased species according to each data characteristic, and determining the probability corresponding to the diseased species according to the characteristic mapping image.
In some embodiments, the processor is further configured to: based on a pre-trained disease classification prediction model, determining disease types and corresponding probability according to each data characteristic and the characteristic mapping image; wherein the disease classification predictive model includes a fully connected layer and a softmax layer.
In some embodiments, the processor is further configured to: and screening each data characteristic according to the characteristic mapping image, and remapping the screened data characteristic to a probability space to obtain a new characteristic mapping image.
The embodiment of the application also provides a computer readable storage medium, and a computer program is stored on the computer readable storage medium, wherein the computer program comprises program instructions, and when the program instructions are executed, any one of the user data processing methods provided by the embodiment of the application is realized.
The computer readable storage medium may be an internal storage unit of the computer device according to the foregoing embodiment, for example, a hard disk or a memory of the computer device. The computer readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), or the like, which are provided on the computer device.
Further, the computer-readable storage medium may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function, and the like; the store data area may store data created from the use of blockchain nodes, and the like.
The invention refers to a novel application mode of computer technologies such as storage, point-to-point transmission, consensus mechanism, encryption algorithm and the like of a blockchain language model. The Blockchain (Blockchain), which is essentially a de-centralized database, is a string of data blocks that are generated in association using cryptographic methods, each of which contains a batch of information for network transactions, for verifying the validity of the information (anti-counterfeiting) and generating the next block. The blockchain may include a blockchain underlying platform, a platform product services layer, an application services layer, and the like.
While the invention has been described with reference to certain preferred embodiments, it will be understood by those skilled in the art that various changes and substitutions of equivalents may be made and equivalents will be apparent to those skilled in the art without departing from the scope of the invention. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (9)
1. A method of user data processing, the method comprising:
acquiring user data of a patient, wherein the user data comprises image data, text data, audio data and structured data;
acquiring a pre-trained decoder corresponding to the image data, the text data and the audio data;
decoding the image data, the text data and the audio data based on the decoder respectively to obtain corresponding data characteristics;
mapping each data characteristic and the structured data to a probability space to obtain a characteristic mapping image;
determining the disease type and the corresponding probability according to each data characteristic and the characteristic mapping image;
the mapping each data feature and the structured data to a probability space to obtain a feature mapping image comprises the following steps:
and mapping each data feature and the structured data to a probability space based on a probability cross-modal embedding model to obtain a feature mapping image.
2. The method of claim 1, wherein decoding the image data, text data, and audio data, respectively, based on the decoder, results in corresponding data features, comprising:
decoding image data based on a ResNet decoder to obtain image characteristics corresponding to the image data;
decoding text data based on a GloVe decoder to obtain text features corresponding to the text data;
and decoding the audio data based on the audio decoder to obtain the audio characteristics corresponding to the audio data.
3. The method of claim 1, wherein the mapping each of the data features and the structured data to a probability space based on a probabilistic cross-modality embedding model results in a feature mapped image, comprising:
mapping the data features corresponding to the user data to a probability space in a single-mode mapping mode to obtain a feature mapping image; or (b)
And mapping the data features corresponding to the user data to a probability space in a multi-mode mapping mode to obtain a feature mapping image.
4. The method of claim 1, wherein said determining the type of illness and the corresponding probability from said each data feature and said feature map image comprises:
and determining the disease type according to each data characteristic, and determining the probability corresponding to the disease type according to the characteristic mapping image.
5. The method of claim 1, wherein said determining the type of illness and the corresponding probability from said each data feature and said feature map image comprises:
based on a pre-trained disease classification prediction model, determining the disease type and the corresponding probability according to each data characteristic and the characteristic mapping image; wherein the disease classification predictive model includes a fully connected layer and a softmax layer.
6. The method of claim 1, wherein said mapping each of said data features and said structured data to a probability space results in a feature mapped image, said method further comprising:
and screening each data characteristic according to the characteristic mapping image, and remapping the screened data characteristic to a probability space to obtain a new characteristic mapping image.
7. A user data processing apparatus, comprising:
the data acquisition module is used for acquiring user data of a patient, wherein the user data comprises image data, text data, audio data and structured data;
the decoder acquisition module is used for acquiring the pre-trained decoder corresponding to the image data, the text data and the audio data;
the feature extraction module is used for respectively decoding the image data, the text data and the audio data based on the decoder to obtain corresponding data features;
the feature mapping module is used for mapping each data feature and the structured data to a probability space to obtain a feature mapping image;
the data processing module is used for determining the disease type and the corresponding probability according to each data characteristic and the characteristic mapping image;
the mapping each data feature and the structured data to a probability space to obtain a feature mapping image comprises the following steps:
and mapping each data feature and the structured data to a probability space based on a probability cross-modal embedding model to obtain a feature mapping image.
8. A computer device, the computer device comprising a memory and a processor;
the memory is used for storing a computer program;
the processor is configured to execute the computer program and implement when executing the computer program:
a user data processing method as claimed in any one of claims 1 to 6.
9. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program which, when executed by a processor, causes the processor to implement the user data processing method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110605578.XA CN113241198B (en) | 2021-05-31 | 2021-05-31 | User data processing method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110605578.XA CN113241198B (en) | 2021-05-31 | 2021-05-31 | User data processing method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113241198A CN113241198A (en) | 2021-08-10 |
| CN113241198B true CN113241198B (en) | 2023-08-08 |
Family
ID=77136002
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110605578.XA Active CN113241198B (en) | 2021-05-31 | 2021-05-31 | User data processing method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113241198B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116578734B (en) * | 2023-05-20 | 2024-04-30 | 重庆师范大学 | Probability embedding combination retrieval method based on CLIP |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109284412A (en) * | 2018-09-20 | 2019-01-29 | 腾讯音乐娱乐科技（深圳）有限公司 | To the method and apparatus of audio data figure |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100211605A1 (en) * | 2009-02-17 | 2010-08-19 | Subhankar Ray | Apparatus and method for unified web-search, selective broadcasting, natural language processing utilities, analysis, synthesis, and other applications for text, images, audios and videos, initiated by one or more interactions from users |
| US11810671B2 (en) * | 2018-12-11 | 2023-11-07 | K Health Inc. | System and method for providing health information |
-
2021
- 2021-05-31 CN CN202110605578.XA patent/CN113241198B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109284412A (en) * | 2018-09-20 | 2019-01-29 | 腾讯音乐娱乐科技（深圳）有限公司 | To the method and apparatus of audio data figure |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113241198A (en) | 2021-08-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Qayyum et al. | Secure and robust machine learning for healthcare: A survey | |
| CN110520872B (en) | Embedded Predictive Machine Learning Model | |
| CN111316281B (en) | Semantic classification method and system for numerical data in natural language context based on machine learning | |
| CN110069779B (en) | Symptom entity identification method of medical text and related device | |
| CN112015917A (en) | Data processing method and device based on knowledge graph and computer equipment | |
| US11468989B2 (en) | Machine-aided dialog system and medical condition inquiry apparatus and method | |
| CN104516942A (en) | Concept driven automatic section identification | |
| CN111666477A (en) | Data processing method and device, intelligent equipment and medium | |
| CN116580849B (en) | Medical data acquisition and analysis system and method thereof | |
| JP6908977B2 (en) | Medical information processing system, medical information processing device and medical information processing method | |
| Agrawal et al. | Health services data: big data analytics for deriving predictive healthcare insights | |
| CN110164519B (en) | Classification method for processing electronic medical record mixed data based on crowd-sourcing network | |
| CN113724830B (en) | Medication risk detection method based on artificial intelligence and related equipment | |
| CN113241198B (en) | User data processing method, device, equipment and storage medium | |
| Rajesh et al. | Machine learning for online automatic prediction of common disease attributes using never-ending image learner | |
| Shi et al. | An automatic classification method on chronic venous insufficiency images | |
| CN113656601A (en) | Doctor-patient matching method, device, equipment and storage medium | |
| CN116741333B (en) | Medicine marketing management system | |
| CN112749277A (en) | Medical data processing method and device and storage medium | |
| Xue et al. | Distance-based detection of cough, wheeze, and breath sounds on wearable devices | |
| CN115036034B (en) | Similar patient identification method and system based on patient characterization map | |
| CN116468043A (en) | Nested entity identification method, device, equipment and storage medium | |
| CN115985506A (en) | Information extraction method and device, storage medium and computer equipment | |
| WO2022249407A1 (en) | Assessment assistance system, assessment assistance method, and recording medium | |
| Aydogan | A hybrid deep neural network‐based automated diagnosis system using x‐ray images and clinical findings |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |