CN112289441B - Medical biological feature information matching system based on multiple modes - Google Patents

Medical biological feature information matching system based on multiple modes Download PDF

Info

Publication number
CN112289441B
CN112289441B CN202011304671.9A CN202011304671A CN112289441B CN 112289441 B CN112289441 B CN 112289441B CN 202011304671 A CN202011304671 A CN 202011304671A CN 112289441 B CN112289441 B CN 112289441B
Authority
CN
China
Prior art keywords
human body
body information
feature
feature set
model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011304671.9A
Other languages
Chinese (zh)
Other versions
CN112289441A (en
Inventor
杜登斌
杜小军
杜乐
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuzheng Intelligent Technology Beijing Co ltd
Original Assignee
Wuzheng Intelligent Technology Beijing Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuzheng Intelligent Technology Beijing Co ltd filed Critical Wuzheng Intelligent Technology Beijing Co ltd
Priority to CN202011304671.9A priority Critical patent/CN112289441B/en
Publication of CN112289441A publication Critical patent/CN112289441A/en
Application granted granted Critical
Publication of CN112289441B publication Critical patent/CN112289441B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Public Health (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Physics & Mathematics (AREA)
  • Molecular Biology (AREA)
  • Medical Informatics (AREA)
  • General Engineering & Computer Science (AREA)
  • Epidemiology (AREA)
  • Pathology (AREA)
  • Primary Health Care (AREA)
  • Databases & Information Systems (AREA)
  • Medical Treatment And Welfare Office Work (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)

Abstract

The invention provides a medical biological characteristic information matching system based on multiple modes. Comprising the following steps: the acquisition module acquires human body information data and corresponding disease information, and respectively establishes a human body information feature set and a disease feature set; the model building module is used for carrying out multi-mode feature fusion according to the human body information feature set and the disease feature set through the convolutional neural network, and building a matching model and a corresponding disease symptom model; the set establishing module is used for acquiring human body information data to be matched and establishing a human body information feature set; and the matching module is used for establishing a typical correlation analysis model, calculating the similarity between the matching model and the human body information feature set according to the typical correlation analysis model, and matching the human body information feature set according to the similarity. The invention can comprehensively and accurately match the medical biological characteristic information through the multi-mode characteristic information, and has the advantages of large matching range, high accuracy and good user experience.

Description

Medical biological feature information matching system based on multiple modes
Technical Field
The invention relates to the field of computers, in particular to a medical biological characteristic information matching system based on multiple modes.
Background
The traditional Chinese medicine is based on syndrome differentiation. Syndrome differentiation, in short, distinguishing the syndrome. The symptoms are different from symptoms, and the symptoms are subjective abnormal sensations or certain pathological changes of patients, such as fever, headache, cough, vomiting, diarrhea and the like; the symptoms are the pathological generalization of the body at a certain stage in the course of disease progression (including symptoms, signs, etc.).
The existing matching of the medical biological characteristic information is often performed manually by a clinician, the matching is single, the multi-mode matching cannot be performed, if the multi-mode information matching is performed, the time is long, the accuracy is low, and therefore improvement of the existing information matching system is needed.
The foregoing is provided merely for the purpose of facilitating understanding of the technical solutions of the present invention and is not intended to represent an admission that the foregoing is prior art.
Disclosure of Invention
In view of the above, the invention provides a multi-mode-based medical biological feature information matching system, which aims to solve the technical problem that the prior art cannot realize the rapid and accurate matching of multi-mode medical biological feature information and disease information through a typical correlation analysis model.
The technical scheme of the invention is realized as follows:
in one aspect, the present invention provides a multi-modality based medical biometric information matching system, comprising:
the acquisition module is used for acquiring human body information data and corresponding disease information, and respectively establishing a human body information feature set and a disease feature set according to the human body information data and the corresponding disease information;
the model building module is used for carrying out multi-mode feature fusion according to the human body information feature set and the disease feature set through the convolutional neural network, and building a matching model and a corresponding disease symptom model;
the set establishing module is used for acquiring the human body information data to be matched, extracting human body information characteristic data from the human body information data to be matched, and establishing a human body information characteristic set according to the human body information characteristic data;
and the matching module is used for establishing a typical correlation analysis model, calculating the similarity between the matching model and the human body information feature set according to the typical correlation analysis model, and matching the human body information feature set according to the similarity.
On the basis of the above technical solution, preferably, the acquiring module includes a preprocessing module, configured to acquire human body information data and corresponding disease information, where the human body information data includes: human body sign image sample information, behavior feature sample information and human body sign text description information, and extracting feature words from human body information data and corresponding disease information.
On the basis of the above technical solution, preferably, the acquiring module includes a feature set establishing module, configured to establish a human body feature image sample feature set, a behavior feature set, a human body feature text description feature set, and a disease feature set according to the human body information data and feature words of the corresponding disease information, respectively, and generate the human body information feature set and correlate the human body feature image sample feature set, the behavior feature set, and the human body feature text description feature set with the disease feature set.
On the basis of the above technical solution, preferably, the model building module includes a convolutional neural computing module, configured to build a neural network model, set a screening condition, select a feature set satisfying the screening condition from a human body information feature set and a disease feature set according to the screening condition as a second feature set, calculate a weight of the second feature set through a full connection layer of the neural network, and build a matching model and a corresponding disease symptom model through the neural network according to the weight.
On the basis of the technical scheme, preferably, the set establishing module comprises a feature extracting module, a data processing module and a data processing module, wherein the feature extracting module is used for setting a data integrity verification standard, acquiring human body information data to be matched, verifying the human body information data to be matched according to the integrity verification standard, extracting human body information feature data from the human body information data to be matched when the human body information data to be matched passes the verification, and establishing a human body information feature set according to the human body information feature data; and when the human body information data to be matched does not pass the verification, re-acquiring the human body information data to be matched.
On the basis of the technical scheme, preferably, the set establishment module comprises a feature set establishment module, wherein the feature set establishment module is used for extracting human body information feature words from human body information data to be matched, clustering the human body information feature words in a clustering mode, and taking the clustered human body information feature words as a human body information feature set.
On the basis of the technical scheme, preferably, the matching module comprises a matching report module, which is used for establishing a typical correlation analysis model, calculating the similarity between the matching model and the human body information feature set according to the typical correlation analysis model, and generating a corresponding information matching report according to the similarity.
Still further preferably, the multi-modality based medical biometric information matching apparatus includes:
the acquisition unit is used for acquiring the human body information data and the corresponding disease information, and respectively establishing a human body information feature set and a disease feature set according to the human body information data and the corresponding disease information;
the model building unit is used for carrying out multi-mode feature fusion according to the human body information feature set and the disease feature set through the convolutional neural network, and building a matching model and a corresponding disease symptom model;
the set establishing unit is used for acquiring the human body information data to be matched, extracting human body information characteristic data from the human body information data to be matched, and establishing a human body information characteristic set according to the human body information characteristic data;
and the matching unit is used for establishing a typical correlation analysis model, calculating the similarity between the matching model and the human body information feature set according to the typical correlation analysis model, and matching the human body information feature set according to the similarity.
Compared with the prior art, the medical biological characteristic information matching system based on the multiple modes has the following beneficial effects:
(1) The multi-mode feature fusion is carried out on the human body information feature set and the disease feature set through the convolutional neural network, so that the depth of information matching is greatly expanded, and the accuracy of information matching is improved;
(2) And calculating the similarity between the matching model and the human body information feature set through the typical correlation analysis model, and improving the accuracy of information matching according to the similarity, and improving the running speed of the whole matching system.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a block diagram of a first embodiment of a multi-modality based medical biometric information matching system of the present invention;
FIG. 2 is a block diagram of a second embodiment of a multi-modality based medical biometric information matching system of the present invention;
FIG. 3 is a block diagram of a third embodiment of a multi-modality based medical biometric information matching system of the present invention;
FIG. 4 is a block diagram of a fourth embodiment of a multi-modality based medical biometric information matching system of the present invention;
FIG. 5 is a block diagram of a fifth embodiment of a multi-modality based medical biometric information matching system of the present invention;
fig. 6 is a block diagram of a medical biometric information matching device based on multiple modes according to the present invention.
Detailed Description
The following description of the embodiments of the present invention will clearly and fully describe the technical aspects of the embodiments of the present invention, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, are intended to fall within the scope of the present invention.
As shown in fig. 1, fig. 1 is a block diagram of a first embodiment of a multi-modality based medical biometric information matching system according to the present invention. Wherein, the medical biological characteristic information matching system based on the multiple modes comprises: the system comprises an acquisition module 10, a model building module 20, a set building module 30 and a matching module 40.
The acquisition module 10 is configured to acquire human body information data and corresponding disease information, and respectively establish a human body information feature set and a disease feature set according to the human body information data and the corresponding disease information;
the model building module 20 is configured to perform multi-mode feature fusion according to the human body information feature set and the disease feature set through the convolutional neural network, and build a matching model and a corresponding disease symptom model;
the set establishing module 30 is configured to acquire human body information data to be matched, extract human body information feature data from the human body information data to be matched, and establish a human body information feature set according to the human body information feature data;
and a matching module 40, configured to establish a typical correlation analysis model, calculate a similarity between the matching model and the human body information feature set according to the typical correlation analysis model, and match the human body information feature set according to the similarity.
Further, as shown in fig. 2, a structural block diagram of a second embodiment of the multi-mode-based medical biometric information matching system according to the present invention is proposed based on the above embodiments, and in this embodiment, the acquisition module 10 further includes:
the preprocessing module 101 is configured to obtain human body information data and corresponding disease information, where the human body information data includes: human body sign image sample information, behavior feature sample information and human body sign text description information, and extracting feature words from human body information data and corresponding disease information.
The preprocessing module 102 is configured to obtain human body information data and corresponding disease information, where the human body information data includes: human body sign image sample information, behavior feature sample information and human body sign text description information, and extracting feature words from human body information data and corresponding disease information.
It should be understood that the human body information data acquired in the present embodiment includes: the method comprises the steps of establishing a human body feature image sample feature set, a behavior feature set, a human body feature text description feature set and a disease feature set respectively according to human body information data and feature words of corresponding disease information, generating the human body information feature set by using the human body feature image sample information, the behavior feature set and the human body feature text description feature set, and correlating the human body information feature set with the disease feature set.
It should be understood that the specific operation steps of this embodiment are: selecting multiple image samples of human body signs (including face, herringbone, nose, ear, acne, toenail, fingerprint, hand shape, tongue head, etc.), extracting abnormal characteristic representation from abnormal image, and obtaining corresponding disease or etiology. Such as looking at the disease through an image of the tongue: 1. the disease is seen from the state image of the tongue. The tongue body is not soft, is inconvenient to bend and stretch, can not even rotate, and most of the tongue body belongs to the symptoms of high heat to hurt body fluid and excessive pathogenic heat, and is also the later stage of heat to hurt yin; the tongue body is not trembled autonomously, and usually belongs to liver wind internal movement or qi and blood deficiency; if the tongue body stretches and contracts, the weakness is often caused by deficiency of qi and blood, loss of yin and yang due to long-term illness, and malnutrition of tendons and vessels; 2. the disease is seen from fat and thin images of the tongue. The tongue is fat and there is tooth marks on the side of the tongue, which are usually related to water retention and usually due to spleen deficiency or kidney-yang deficiency. Most spleen deficiency reflects pathological changes in the digestive system, poor gastrointestinal motility and poor absorption. Deficiency of kidney yang is accompanied by cold limbs, soreness and weakness of waist and knees, abnormal urination and defecation; the thin tongue body is generally in deficiency syndrome, and if the tongue body is light in color, the symptoms of hypodynamia, laziness in speaking and lassitude appear; a deep red tongue is usually due to yin deficiency and internal heat, and is generally restless and sweats; 3. the disease is seen from the barbed image of the tongue. Sometimes, the hyperplasia and hypertrophy of the tongue tip and the tongue nipple at the tongue edge can be seen, and the protrusion is thorn, which is caused by damp-heat in liver and gallbladder or excessive heart fire; if the heat evil is greater, the larger and more the barbed points are. All females with the tongue thorns are heat evil for many days, consume yin fluid, and can have symptoms of sore throat, body surface furuncle, dry stool and the like; 4. the disease was seen from the cracked images of the tongue. The tongue body has various transverse or longitudinal fissures or wrinkles, and the tongue is dark red and cracked, which is caused by in vivo heat, and is usually accompanied by halitosis, bitter taste and sore throat; if the tongue is light in color and has cracks, the tongue is mostly deficient in breath and yin, and is accompanied by dry mouth and throat, palpitation and shortness of breath. A few normal people have cracked tongues, and the cracks do not change with the health condition of the body, exist all the year round, and the like.
The system can also obtain multiple samples of behavior characteristics (such as body temperature change, morphological change, smell change, color change, sound change, etc., and sweating, bleeding, acne, speckle, etc.), and extract characteristic representations of abnormal behaviors respectively to obtain corresponding diseases or causes. Taking the abnormal smell of abnormal behavior as an example: when diabetes patients suffer from ketoacidosis, apple-like sweetness can be exhaled; various organophosphorus pesticides poisoning people can exhale a common and special garlic-like smell; the mistaking of the rodenticide zinc phosphide or excessive eating of the shallot and garlic can also give out the same smell throughout the body, particularly in the oral cavity; the patient with typhoid fever can smell the bread; beer taste is provided for patients with scrofula; for cyanide poisoning, there is almond taste in the mouth; when the tanning worker is poisoned by hydrogen sulfide, the tanning worker can exhale the foul smell of the egg; patients with severe liver function impairment and hepatic coma often develop a special mouse odor in the exhaled air, which is called liver odor; pork smell is present in yellow fever patients; bromhidrosis is often emitted by patients with bromhidrosis; chicken feather flavor and the like can be emitted on the measles patient.
Finally, the system can acquire the text description of the human body symptoms, extract the abnormal characteristic representation and acquire the corresponding diseases or causes. Such as: 1. the heart fire is divided into deficiency and excess, and the deficiency fire is manifested as low fever, night sweat, vexation, dry mouth, etc.; excessive fire manifests as recurrent oral ulceration, dry mouth, scanty and reddish urine, restlessness and irritability; 2. kidney fire is mainly manifested as dizziness, tinnitus, deafness, loss of teeth, restlessness, feverish sensation in the chest, abdomen, emaciation, soreness of waist and legs, etc.; 3. lung fire. For those with excessive lung fire, the symptoms of dry mouth and tongue, sore throat, cough and hemoptysis are likely to occur; 4. liver fire manifests as headache, redness of the face and eyes, dry mouth and throat pain, hypochondriac pain, yellow urine, constipation, and even hematemesis.
Further, as shown in fig. 3, a structural block diagram of a third embodiment of the multi-modal-based medical biometric information matching system according to the present invention is provided based on the above embodiments, and in this embodiment, the model building module 20 further includes:
the convolutional neural calculation module 201 is configured to establish a neural network model, set screening conditions, select a feature set satisfying the screening conditions from the human body information feature set and the disease feature set according to the screening conditions as a second feature set, calculate weights of the second feature set through a full connection layer of the neural network, and establish a matching model and a corresponding disease symptom model through the neural network according to the weights.
It should be understood that after obtaining the human body information feature set, the system sets a screening condition, and selects a feature set satisfying the screening condition from the human body information feature set and the disease feature set as a second feature set according to the screening condition, calculates a weight of the second feature set through a full connection layer of the neural network, and establishes a matching model and a corresponding disease symptom model through the neural network according to the weight, wherein the screening condition is set by an administrator.
It should be understood that the specific operational flow is as follows: and (3) carrying out multi-modal feature fusion by using a convolutional neural network method, and establishing an estimation model, a disease and etiology presumption model and a traditional Chinese medicine prescription recommendation model of the semantic similarity of the abnormal physical sign image features of the human body, the abnormal behavior features of the human body and the text description features of the abnormal signs of the human body. Here, according to the correlation between different modes, screening out the characteristics meeting the preset conditions from the first characteristic set of each mode to obtain the second characteristic set of each mode; and determining the weight of the second feature set of each mode at the full connection layer of the multi-mode convolutional neural network, and fusing the second feature sets of the modes according to the weight so that the fused second feature set trains the multi-mode convolutional neural network for biological feature recognition. Let f 1 Image feature set f representing abnormal physical sign of human body 2 Representing abnormal behavior feature set, f of human body 3 A set of textual descriptive features representing signs of a human anomaly. Inputting the features in each feature set into a multi-mode feature convolution learning layer to obtain three corresponding feature matrixes and dividing the feature matrixesThe method comprises the following steps: w (W) 1 、W 2 、W 3 Then W is determined according to the weight of different modes 1 f 1 、W 2 f 2 、W 3 f 3 And (5) carrying out series connection. The weight determination may be preset by a technician, or may be determined according to a training result in the training process. Therefore, the multimode convolutional neural network for feature recognition is obtained by fusing the multimode features and training the multimode convolutional neural network according to the fused features, so that the problem of limitation of single-mode recognition in the prior art can be solved, the accuracy of biological feature recognition is improved, and the recognition of multimode tasks under a complex scene under a controlled condition is realized.
Further, as shown in fig. 4, a structural block diagram of a fourth embodiment of the multi-modality-based medical biometric information matching system of the present invention is proposed based on the above embodiments, and in this embodiment, the set-up module 30 includes:
the feature extraction module 301 is configured to set a data integrity verification standard, obtain human body information data to be matched, verify the human body information data to be matched according to the integrity verification standard, extract human body information feature data from the human body information data to be matched when the human body information data to be matched passes the verification, and establish a human body information feature set according to the human body information feature data; and when the human body information data to be matched does not pass the verification, re-acquiring the human body information data to be matched.
The feature set establishing module 302 is configured to extract human body information feature words from human body information data to be matched, cluster the human body information feature words in a clustering manner, and use the clustered human body information feature words as a human body information feature set.
It should be understood that, after that, the system sets a data integrity verification standard, acquires the human body information data to be matched, verifies the human body information data to be matched according to the integrity verification standard, extracts human body information feature data from the human body information data to be matched when the human body information data to be matched passes verification, extracts human body information feature words from the human body information data to be matched, clusters the human body information feature words in a clustering mode, and uses the clustered human body information feature words as a human body information feature set; and when the human body information data to be matched does not pass the verification, re-acquiring the human body information data to be matched.
Further, as shown in fig. 5, a structural block diagram of a fifth embodiment of the multi-modality-based medical biometric information matching system according to the present invention is proposed based on the above embodiments, and in this embodiment, the matching module 40 includes:
the matching report module 401 is configured to establish a typical correlation analysis model, calculate a similarity between the matching model and the human body information feature set according to the typical correlation analysis model, and generate a corresponding information matching report according to the similarity.
It should be understood that the final system will build a model of the canonical correlation analysis by which the similarity between the matching model and the set of human information features is calculated, from which a corresponding information matching report is generated.
It should be appreciated that classical correlation analysis is a multivariate statistical approach to study the correlation of two sets of variables. In general, principal Component Analysis (PCA) is the linear combination of the original correlated variables into independent variables (projections) to make use of the principal component variables for more efficient analysis. Whereas the idea of a typical correlation analysis (CCA) is: analysis of independent variable group x= [ X ] 1 ,x 2 ,x 3 ,...,x p ]Dependent variable group y= [ Y ] 1 ,y 2 ,y 3 ,...,y p ]Correlation between them. (each argument X of X here 1 Is a column vector representing a plurality of observations). If the conventional correlation analysis is adopted, only the correlation coefficient of each variable of X and each variable of Y is required, so that a correlation coefficient matrix R= [ R ] is formed ij ]p*q,r ij Represents the ith argument x i With the j-th dependent variable y j Correlation coefficient between the two. However, this is disadvantageous: only the relation between X and Y is roughly considered, but the possible correlation between X independent variables is ignored, as is the case between Y dependent variables. Method class of resolutionSimilar to principal component analysis, we can extract X as the principal component and Y as the principal component, so that the linearity inside X, Y is uncorrelated, thus the above-mentioned drawbacks are solved by using principal components to study the correlation between X and Y.
It should be understood that, specifically, in order to construct a typical correlation analysis model, a conventional typical correlation analysis is first used to obtain the similarity between the text description feature of a specific sign anomaly image feature, a corresponding behavior feature and its accompanying sign and the text description feature of the specific sign anomaly image feature to be identified, a corresponding behavior feature and its accompanying sign. And constructing a typical correlation analysis model according to the parameters obtained by solving. The canonical correlation analysis model may be a closely ordered canonical correlation analysis model or a multi-row canonical correlation analysis model.
In this embodiment, the specific steps are:
1. selecting a specific sign image with a matching relationship, corresponding behavior characteristics and text description of accompanying signs thereof as reference data;
2. respectively calculating the characteristics of a specific sign image, the corresponding behavior characteristics and the text description of the accompanying signs;
3. calculating the distance between a specific sign image to be identified and each reference sign by using the sign image features as new features;
4. calculating the distance between a specific behavior feature to be identified and each reference behavior feature by using the behavior features to serve as new features;
5. and obtaining a similarity estimation function among the abnormal image features of a specific sign, the corresponding behavior features and the text description features of the accompanying sign by carrying out correlation analysis or typical correlation analysis on the reference distance features.
It should be understood that, finally, the system performs disease cognition and Chinese medicinal prescription recommendation according to the results obtained by the estimation model and the disease and etiology presumption model of the semantic similarity of the abnormal physical sign image features of the human body, the abnormal behavior features of the human body and the text description features of the abnormal signs of the human body and the Chinese medicinal prescription recommendation model.
It should be understood that the Chinese medicinal prescription recommendation model is systematically obtained from a Chinese medicinal prescription adaptation library, which is a locally existing database, and can also be network data, for example, a Chinese medicinal prescription for treating insomnia: 1. liver fire hyperactivity is superior to heart, mainly with liver-fire discharging and tranquillizing effects, the recipe name is liver-fire discharging and tranquillizing decoction, which is prepared from thirty grams of raw nacre, fifteen grams of uncaria, fifteen grams of red sage root, fifteen grams of selfheal, ten grams of cinnabar and ten grams of cortex albiziae by decocting with water, and is taken twice daily; 2. the prescription name Shu Anshang mainly comprises twenty-four grams of fried jujube kernels, fifteen grams of fried platycladi seeds, fifteen grams of tuber fleeceflower stems, fifteen grams of raw keels, nine grams of dried rehmannia root and six grams of red sage roots, and is decocted with water for twice daily administration, thus having good effect on treating insomnia. For another example, the prescription of the traditional Chinese medicine for treating cervical spondylosis: the cervical spondylosis caused by wind-cold exogenous infection can be selected from radix puerariae soup; the decoction for removing blood stasis is used for treating cervical spondylosis caused by blood stasis; the cervical spondylosis caused by the wind-cold-dampness arthralgia can be modified by using nine-ingredient notopterygium root decoction; for cervical spondylosis due to liver wind, it can be modified by Tian Gao Gou Teng Yin.
It should be noted that the foregoing is merely illustrative, and does not limit the technical solutions of the present application in any way.
As can be easily found from the above description, the present embodiment proposes a medical biometric information matching system based on multiple modes, including: the acquisition module is used for acquiring human body information data and corresponding disease information, and respectively establishing a human body information feature set and a disease feature set according to the human body information data and the corresponding disease information; the model building module is used for carrying out multi-mode feature fusion according to the human body information feature set and the disease feature set through the convolutional neural network, and building a matching model and a corresponding disease symptom model; the set establishing module is used for acquiring the human body information data to be matched, extracting human body information characteristic data from the human body information data to be matched, and establishing a human body information characteristic set according to the human body information characteristic data; and the matching module is used for establishing a typical correlation analysis model, calculating the similarity between the matching model and the human body information feature set according to the typical correlation analysis model, and matching the human body information feature set according to the similarity. According to the embodiment, the medical biological characteristic information can be comprehensively and accurately matched through the multi-mode characteristic information, the matching range is large, the accuracy is high, and the user experience is good.
In addition, the embodiment of the invention also provides medical biological characteristic information matching equipment based on multiple modes. As shown in fig. 6, the multi-modality-based medical biometric information matching apparatus includes: an acquisition unit 10, a model creation unit 20, a set creation unit 30, and a matching unit 40.
An acquiring unit 10, configured to acquire human body information data and corresponding disease information, and respectively establish a human body information feature set and a disease feature set according to the human body information data and the corresponding disease information;
the model building unit 20 is configured to perform multi-mode feature fusion according to the human body information feature set and the disease feature set through the convolutional neural network, and build a matching model and a corresponding disease symptom model;
a set establishing unit 30, configured to acquire human body information data to be matched, extract human body information feature data from the human body information data to be matched, and establish a human body information feature set according to the human body information feature data;
and a matching unit 40 for establishing a typical correlation analysis model, calculating the similarity between the matching model and the human body information feature set according to the typical correlation analysis model, and matching the human body information feature set according to the similarity.
In addition, it should be noted that the above embodiment of the apparatus is merely illustrative, and does not limit the scope of the present invention, and in practical application, a person skilled in the art may select some or all modules according to actual needs to achieve the purpose of the embodiment, which is not limited herein.
In addition, technical details not described in detail in the present embodiment may refer to the multi-mode-based medical biometric information matching system provided in any embodiment of the present invention, which is not described herein.
The foregoing description of the preferred embodiments of the invention is not intended to be limiting, but rather is intended to cover all modifications, equivalents, alternatives, and improvements that fall within the spirit and scope of the invention.

Claims (4)

1. A multi-modality based medical biometric information matching system, the multi-modality based medical biometric information matching system comprising:
the acquisition module is used for acquiring human body information data and corresponding disease information, and respectively establishing a human body information feature set and a disease feature set according to the human body information data and the corresponding disease information;
the acquisition module comprises a preprocessing module and is used for acquiring human body information data and corresponding disease information, wherein the human body information data comprises: human body sign image sample information, behavior feature sample information and human body sign text description information, and extracting feature words from human body information data and corresponding disease information;
the model building module is used for carrying out multi-mode feature fusion according to the human body information feature set and the disease feature set through the convolutional neural network, and building a matching model and a corresponding disease symptom model;
the model building module comprises a convolutional neural calculation module, a model matching module and a disease symptom model, wherein the convolutional neural calculation module is used for building a neural network model, setting screening conditions, selecting a characteristic set meeting the screening conditions from a human body information characteristic set and a disease characteristic set according to the screening conditions as a second characteristic set, calculating the weight of the second characteristic set through a full-connection layer of the neural network, and building the matching model and the corresponding disease symptom model through the neural network according to the weight;
carrying out multi-modal feature fusion by utilizing a convolutional neural network method, and establishing an estimation model, a disease and etiology presumption model and a traditional Chinese medicine prescription recommendation model of semantic similarity of abnormal physical sign image features of a human body, abnormal behavior features of the human body and text description features of abnormal signs of the human body;
the set establishing module is used for acquiring the human body information data to be matched, extracting human body information characteristic data from the human body information data to be matched, and establishing a human body information characteristic set according to the human body information characteristic data;
the set establishing module comprises a feature extracting module, a data processing module and a data processing module, wherein the feature extracting module is used for setting a data integrity verification standard, acquiring human body information data to be matched, verifying the human body information data to be matched according to the integrity verification standard, extracting human body information feature data from the human body information data to be matched when the human body information data to be matched passes the verification, and establishing a human body information feature set according to the human body information feature data; when the human body information data to be matched does not pass the verification, the human body information data to be matched is acquired again;
the matching module is used for establishing a typical correlation analysis model, calculating the similarity between the matching model and the human body information feature set according to the typical correlation analysis model, matching the human body information feature set according to the similarity, and generating a corresponding information matching report, disease cognition and Chinese medicine prescription recommendation.
2. The multi-modality based medical biometric information matching system of claim 1, wherein: the acquisition module comprises a feature set establishment module which is used for respectively establishing a human body feature image sample feature set, a behavior feature set, a human body feature text description feature set and a disease feature set according to the human body information data and the feature words of the corresponding disease information, and generating the human body information feature set by the human body feature image sample feature set, the behavior feature set and the human body feature text description feature set and correlating with the disease feature set.
3. The multi-modality based medical biometric information matching system of claim 1, wherein: the set establishment module comprises a feature set establishment module and is used for extracting human body information feature words from human body information data to be matched, clustering the human body information feature words in a clustering mode, and taking the clustered human body information feature words as a human body information feature set.
4. A multi-modality based medical biometric information matching apparatus, characterized in that the multi-modality based medical biometric information matching apparatus comprises:
the acquisition unit is used for acquiring the human body information data and the corresponding disease information, and respectively establishing a human body information feature set and a disease feature set according to the human body information data and the corresponding disease information;
the model building unit is used for carrying out multi-mode feature fusion according to the human body information feature set and the disease feature set through the convolutional neural network, and building a matching model and a corresponding disease symptom model;
the set establishing unit is used for acquiring the human body information data to be matched, extracting human body information characteristic data from the human body information data to be matched, and establishing a human body information characteristic set according to the human body information characteristic data;
and the matching unit is used for establishing a typical correlation analysis model, calculating the similarity between the matching model and the human body information feature set according to the typical correlation analysis model, and matching the human body information feature set according to the similarity.
CN202011304671.9A 2020-11-19 2020-11-19 Medical biological feature information matching system based on multiple modes Active CN112289441B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011304671.9A CN112289441B (en) 2020-11-19 2020-11-19 Medical biological feature information matching system based on multiple modes

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011304671.9A CN112289441B (en) 2020-11-19 2020-11-19 Medical biological feature information matching system based on multiple modes

Publications (2)

Publication Number Publication Date
CN112289441A CN112289441A (en) 2021-01-29
CN112289441B true CN112289441B (en) 2024-03-22

Family

ID=74399513

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011304671.9A Active CN112289441B (en) 2020-11-19 2020-11-19 Medical biological feature information matching system based on multiple modes

Country Status (1)

Country Link
CN (1) CN112289441B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112971718A (en) * 2021-02-05 2021-06-18 北京鹰之眼智能健康科技有限公司 Syndrome identification method and device, electronic equipment and storage medium
CN112992378B (en) * 2021-04-25 2021-09-03 南京大经中医药信息技术有限公司 Multi-dimensional traditional Chinese medicine prescription recommendation method and recommendation system
CN113488183B (en) * 2021-06-30 2023-10-31 吾征智能技术(北京)有限公司 Heating disease multi-mode feature fusion cognitive system, equipment and storage medium
CN116524248B (en) * 2023-04-17 2024-02-13 首都医科大学附属北京友谊医院 Medical data processing device, method and classification model training device
CN116682526B (en) * 2023-08-03 2023-10-24 中国中医科学院中国医史文献研究所 Traditional Chinese medicine knowledge recommendation system based on ancient book knowledge unit processing

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2010042947A2 (en) * 2008-10-10 2010-04-15 Cardiovascular Decision Technologies, Inc. Automated management of medical data using expert knowledge and applied complexity science for risk assessment and diagnoses
CN107910060A (en) * 2017-11-30 2018-04-13 百度在线网络技术(北京)有限公司 Method and apparatus for generating information
KR101959438B1 (en) * 2018-08-06 2019-03-18 전북대학교 산학협력단 Medical image diagnosis system using multi-modality image creation technique
CN111091906A (en) * 2019-10-31 2020-05-01 中电药明数据科技(成都)有限公司 Auxiliary medical diagnosis method and system based on real world data
CN111863240A (en) * 2020-07-08 2020-10-30 中润普达(十堰)大数据中心有限公司 Disease cognitive system based on abnormal change of human body fluid
CN117012370A (en) * 2023-08-09 2023-11-07 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) Multi-mode disease auxiliary reasoning system, method, terminal and storage medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2010042947A2 (en) * 2008-10-10 2010-04-15 Cardiovascular Decision Technologies, Inc. Automated management of medical data using expert knowledge and applied complexity science for risk assessment and diagnoses
CN107910060A (en) * 2017-11-30 2018-04-13 百度在线网络技术(北京)有限公司 Method and apparatus for generating information
KR101959438B1 (en) * 2018-08-06 2019-03-18 전북대학교 산학협력단 Medical image diagnosis system using multi-modality image creation technique
CN111091906A (en) * 2019-10-31 2020-05-01 中电药明数据科技(成都)有限公司 Auxiliary medical diagnosis method and system based on real world data
CN111863240A (en) * 2020-07-08 2020-10-30 中润普达(十堰)大数据中心有限公司 Disease cognitive system based on abnormal change of human body fluid
CN117012370A (en) * 2023-08-09 2023-11-07 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) Multi-mode disease auxiliary reasoning system, method, terminal and storage medium

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Analysis of the Therapeutic Effect of Multimode Mechanical Thrombectomy in the Treatment of Acute Ischemic Stroke;Li-Na Zhang,等;《world neurosurgery》;第488-493页 *
一种基于DNA 计算的多模态函数求解模型;张友鹏,等;《铁道学报》;第27卷(第6期);第112-116页 *
国外人工智能辅助自闭症谱系障碍诊断及其康复训练研究进展;马博森,等;《康复学报》;第65-69页 *

Also Published As

Publication number Publication date
CN112289441A (en) 2021-01-29

Similar Documents

Publication Publication Date Title
CN112289441B (en) Medical biological feature information matching system based on multiple modes
Hassantabar et al. CovidDeep: SARS-CoV-2/COVID-19 test based on wearable medical sensors and efficient neural networks
JP5400217B2 (en) Computer-aided diagnosis system for determining skin composition based on the principles of traditional Chinese medicine (TCM)
CN108154934A (en) Database Systems based on intelligent traditional Chinese medical science robot
CN109903836A (en) A kind of diet intelligent recommendation and matching system and method based on constitution and big data
CN107887024A (en) Intelligent diagnosis system and traditional Chinese medical science robot based on traditional Chinese medical science robot
CN109300123A (en) Physical analysis method based on face and tongue Image Acquisition
CN109166619A (en) Chinese medicine intelligent diagnostics auxiliary system and method based on neural network algorithm
CN1804850A (en) Symptom and sign differentiation, diagnosis and treatment system in traditional Chinese medicine
CN108877923A (en) A method of the tongue fur based on deep learning generates prescriptions of traditional Chinese medicine
CN114842314B (en) Tongue picture identification method and device based on neural network and computer equipment
TW202018728A (en) Multidimensional health concept guidance system based on traditional chinese medicine theory
CN110321827A (en) A kind of pain level appraisal procedure based on face pain expression video
US20240008813A1 (en) Smart wearable device and method for estimating traditional medicine system parameters
CN112002419B (en) Disease auxiliary diagnosis system, equipment and storage medium based on clustering
Lu et al. Pulse feature extraction based on improved gaussian model
CN112386245A (en) Method for analyzing body condition from tongue
CN112233804B (en) Intelligent recommendation method for warp and equipment applying method
Bakshi et al. Introduction about traditional Tongue Diagnosis with scientific value addition
CN112086200B (en) System, equipment and storage medium for predicting diseases based on bitter taste
CN212809784U (en) Personal health management system based on image recognition and big data
Tasli et al. Integrating remote PPG in facial expression analysis framework
Lin et al. Automatic sublingual vein feature extraction system
CN112270186B (en) Mouth based on entropy model peppery text information matching system
CN111599468A (en) Personal health management system based on image recognition and big data

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant