CN114121269A - Traditional Chinese medicine facial diagnosis auxiliary diagnosis method and device based on face feature detection and storage medium - Google Patents
Traditional Chinese medicine facial diagnosis auxiliary diagnosis method and device based on face feature detection and storage medium Download PDFInfo
- Publication number
- CN114121269A CN114121269A CN202210093004.3A CN202210093004A CN114121269A CN 114121269 A CN114121269 A CN 114121269A CN 202210093004 A CN202210093004 A CN 202210093004A CN 114121269 A CN114121269 A CN 114121269A
- Authority
- CN
- China
- Prior art keywords
- face
- feature
- diagnosis
- key
- facial
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
- G16H20/90—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to alternative medicines, e.g. homeopathy or oriental medicines
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Public Health (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Theoretical Computer Science (AREA)
- Primary Health Care (AREA)
- Epidemiology (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Pharmacology & Pharmacy (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Alternative & Traditional Medicine (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Databases & Information Systems (AREA)
- Pathology (AREA)
- Measuring And Recording Apparatus For Diagnosis (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a traditional Chinese medicine face diagnosis auxiliary diagnosis method based on face feature detection, which comprises the steps of collecting a face image, and labeling face diagnosis key features of the face image for face feature detection model training, wherein the face diagnosis key features comprise at least one of spots, nevi, acne, blood streak and transverse striation; acquiring a face region required by facial diagnosis by adopting a face key point detection algorithm, and carrying out traditional Chinese medicine facial diagnosis partitioning on the face image according to the face key point positioning; extracting the key features of the facial diagnosis from the face image through a convolutional neural network structure, wherein the convolutional neural network structure comprises at least one convolutional layer, one pooling layer, one feature fusion and one feature pyramid; and (4) combining the key characteristics of the facial diagnosis and the traditional Chinese medicine facial diagnosis subareas, and obtaining a traditional Chinese medicine facial diagnosis result according to medical logic judgment. The invention can automatically extract key features of human face such as spots, moles, acne, striations and the like, and effectively support the improvement of the accuracy of the traditional Chinese medicine facial diagnosis.
Description
Technical Field
The present disclosure relates to the field of computer software technologies, and in particular, to a method, an apparatus, a storage medium, and an electronic device for performing a facial diagnosis based on human face feature detection.
Background
The facial diagnosis refers to the diagnosis of visceral diseases and health conditions through the facial reflex zones, so as to cure the disease rapidly. The method is a simple and effective method for discovering diseases, and has very important significance for medicine. The traditional Chinese medicine observes the whole face and five sense organs of the face by four diagnostic methods of inspection, auscultation, inquiry and excision, thereby judging the pathological changes of the whole body and the local parts of the human body. Abnormal changes of the body may occur before or after the facial changes occur, the functions of internal organs and qi and blood conditions of the human body are expressed on the face, and the health state and the disease changes of the human body can be known according to the changes of muscle tension, elasticity and contractility, as well as facial features such as swelling, wrinkles, scabbing, defects, facial skin color changes, congestion and the like by checking the face. When a plurality of traditional Chinese medicine facial diagnosis instruments perform facial diagnosis, a small amount of facial feature information such as color, luster and the like of a facial region is mostly extracted, and the accuracy and reliability of the extracted features are not high, so that the accuracy of the facial diagnosis result is seriously influenced. Because the extracted face features are single, face feature information such as face spots, moles, acne, blood streak, horizontal lines and the like is not utilized, the actual condition of a patient cannot be accurately reflected by a face diagnosis result, the face diagnosis result is mostly single face, the reliability of the diagnosis result is not high, and the wide spread and application of the face diagnosis in the general public are limited.
Disclosure of Invention
In order to solve the above problems, embodiments of the present specification provide a method, an apparatus, a storage medium, and an electronic device for traditional Chinese medicine diagnosis based on human face feature detection.
In order to solve the above technical problem, the embodiments of the present specification are implemented as follows:
in a first aspect, a method for traditional Chinese medicine facial diagnosis auxiliary diagnosis based on human face feature detection is provided, which comprises the following steps:
collecting a face image;
marking the key facial features of the face image for training a face feature detection model, wherein the key facial features comprise at least one of spots, moles, acne, blood streak and transverse striations;
acquiring a face region required by facial diagnosis by adopting a face key point detection algorithm, and carrying out traditional Chinese medicine facial diagnosis partitioning on the face image according to the face key point positioning;
extracting the key features of the facial diagnosis from the face image through a convolutional neural network structure, wherein the convolutional neural network structure comprises at least one convolutional layer, one pooling layer, one feature fusion and one feature pyramid;
and (4) combining the key characteristics of the facial diagnosis and the traditional Chinese medicine facial diagnosis subareas, and obtaining a traditional Chinese medicine facial diagnosis result according to medical logic judgment.
In a second aspect, a facial diagnosis auxiliary diagnosis device for traditional Chinese medicine based on human face feature detection is provided, which includes:
the face image acquisition module is used for acquiring a face image;
the model training module is used for training a face feature detection model according to face diagnosis key features marked on the face image, wherein the face diagnosis key features comprise at least one of spots, moles, acne, blood streak and horizontal lines;
the face image partitioning module is used for acquiring a face region required by face diagnosis by adopting a face key point detection algorithm so as to perform traditional Chinese medicine face diagnosis partitioning on the face image according to the face key point positioning;
the face key feature extraction module is used for extracting the face diagnosis key features from the face image through a convolutional neural network structure, wherein the convolutional neural network structure comprises at least one convolution layer, one pooling layer, one feature fusion and one feature pyramid;
and the facial diagnosis result output module is used for combining the facial diagnosis key characteristics and the traditional Chinese medicine facial diagnosis subareas and obtaining a traditional Chinese medicine facial diagnosis result according to medical logic judgment.
In a third aspect, an electronic device is provided, including: a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to perform the method of the first aspect.
In a fourth aspect, a computer-readable storage medium is presented, storing one or more programs which, when executed by an electronic device comprising a plurality of application programs, cause the electronic device to perform the method of the first aspect.
The specification can achieve at least the following technical effects:
the visible light face image collected by the invention uses the convolutional neural network to automatically extract the key characteristics of spots, moles, acne, blood streak, transverse striations and the like of the face image required by the traditional Chinese medicine face diagnosis, combines the automatic partition of the face image, and obtains the traditional Chinese medicine face diagnosis result according to the medical logic judgment, thereby effectively improving the accuracy of the traditional Chinese medicine face diagnosis.
Drawings
In order to more clearly illustrate the embodiments of the present specification or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only some embodiments described in the present specification, and for those skilled in the art, other drawings can be obtained according to the drawings without any creative effort.
Fig. 1 is a schematic diagram illustrating a step of a traditional Chinese medicine facial diagnosis auxiliary diagnosis method based on face feature detection according to an embodiment of the present disclosure.
Fig. 2 is a second schematic step diagram of a facial diagnosis assistant diagnosis method based on human face feature detection in the embodiment of the present specification.
Fig. 3 is a third schematic step diagram of a facial diagnosis assistance method based on face feature detection in the embodiment of the present specification.
Fig. 4 is a fourth schematic step diagram of a traditional Chinese medicine facial diagnosis auxiliary diagnosis method based on face feature detection provided in the embodiment of the present specification.
Fig. 5 is a schematic structural diagram of a facial diagnosis assistance device in traditional Chinese medicine based on face feature detection according to an embodiment of the present disclosure.
Fig. 6 is a schematic structural diagram of an electronic device provided in an embodiment of the present specification.
Detailed Description
In order to make those skilled in the art better understand the technical solutions in the present specification, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is obvious that the described embodiments are only a part of the embodiments of the present specification, and not all of the embodiments. All other embodiments obtained by a person skilled in the art based on the embodiments in the present specification without any inventive step should fall within the scope of protection of the present specification.
Key terms
A convolutional neural network: the method is a feedforward neural network which comprises convolution calculation and has a deep structure, and is one of representative algorithms of deep learning. The convolutional neural network has the characteristic learning ability, can perform translation invariant classification on input information according to the hierarchical structure of the convolutional neural network, and is applied to the fields of computer vision, natural language processing and the like. The convolutional neural network includes an input layer, a hidden layer, and an output layer. The input layer may process multidimensional data, and typically, the input layer of a one-dimensional convolutional neural network receives a one-dimensional or two-dimensional array, where the one-dimensional array is typically a time or frequency spectrum sample; the two-dimensional array may include a plurality of channels; an input layer of the two-dimensional convolutional neural network receives a two-dimensional or three-dimensional array; the input layer of the three-dimensional convolutional neural network receives a four-dimensional array. The hidden layer comprises a convolution layer, a pooling layer and a full-connection layer 3 common structures; the convolutional layer and the pooling layer are specific to the convolutional neural network, the convolutional layer is mainly used for extracting image features, higher-order image features can be obtained through multiple stacking, and the pooling layer is mainly used for feature dimension reduction, data and parameter quantity compression, overfitting is reduced, and meanwhile fault tolerance of the model is improved.
A characteristic pyramid: is a fundamental component in recognition systems for detecting objects of different dimensions. The identification of the target on multiple scales is a challenge of computer vision, the model accuracy is improved by extracting and fusing multi-scale feature information, and the method can be used in various applications such as target detection, instance segmentation, gesture identification, face identification and the like. Particularly, the feature pyramid can predict by fusing the features of different layers by simultaneously utilizing high resolution of the low-layer features and high-order semantic information of the high-layer features, and the performance of small target detection is greatly improved under the condition that the calculation amount of the original model is not increased basically.
The following describes a specific example of a traditional Chinese medicine diagnosis assisting scheme based on face feature detection in the present specification.
Example one
Referring to fig. 1, a schematic diagram of steps of a traditional Chinese medicine facial diagnosis auxiliary diagnosis method based on face feature detection provided in an embodiment of the present specification is shown. The method may comprise the steps of:
step 101: and collecting a human face image. Specifically, the acquisition of the face image is the basis of the auxiliary diagnosis of the traditional Chinese medicine facial diagnosis, and the visible light face image can be acquired by the camera to be used as the original image of the traditional Chinese medicine facial diagnosis.
Step 102: and marking the key features of the facial diagnosis of the face image for training a face feature detection model, wherein the key features of the facial diagnosis comprise at least one of spots, moles, acne, blood streak and transverse striations. It should be noted that the key features of the facial diagnosis can be flexibly selected according to the scientific analysis of the body part and diseases reflected by the facial features of the facial diagnosis in traditional Chinese medicine, and are not limited to the spots, moles, pox, blood streak and transverse striation. That is, all the key characteristics of the face diagnosis that can assist the face diagnosis of the traditional Chinese medicine and improve the accuracy of the disease diagnosis are also applicable to the scheme of the invention without violating the scheme of the invention.
Step 103: and acquiring a face region required by facial diagnosis by adopting a face key point detection algorithm, and carrying out traditional Chinese medicine facial diagnosis partitioning on the face image according to the face key point positioning.
Optionally, the traditional Chinese medicine diagnosis areas at least comprise a heart area, a lung area, a liver area, a spleen area, a gallbladder area and a kidney area.
Optionally, when the face region is obtained, a face key point detection algorithm needs to be adopted. In one embodiment of the invention, a face key point detection algorithm of the dlib library is adopted. The dlib is an open source library for machine learning, contains a plurality of algorithms for machine learning, is convenient to use, and directly contains a header file, and does not depend on other libraries because the dlib is provided with image coding and decoding library source codes. dlib can create many sophisticated machine learning aspects of software to help solve practical problems. Dlib is currently widely used in industry and academic fields, including robotics, embedded devices, mobile phones and large high-performance computing environments. The working principle of the face key point detection algorithm carried by the Dlib library comprises the following steps: extracting feature points, obtaining a feature data set, writing the feature data set into the csv, and calculating the Euclidean distance of the feature data set for comparison. In the present embodiment, 81 key points of a human face are extracted for detection. It should be noted here that the number of extracted key points is related to the accuracy of the detection result, so that all the key points can assist the traditional Chinese medical diagnosis and improve the accuracy of disease diagnosis, and different numbers of key points can be selected for detection without violating the scheme of the present invention, which is also applicable to the scheme of the present invention.
Step 104: the face image extracts the key features of the face diagnosis through a convolutional neural network structure, and the convolutional neural network structure comprises at least one convolutional layer, one pooling layer, one feature fusion and one feature pyramid.
It should be noted that the convolutional layer is mainly used for extracting image features, and higher-order image features can be obtained through multiple stacking; the pooling layer is mainly used for feature dimension reduction, data and parameter quantity compression, overfitting reduction and model fault tolerance improvement; upsampling is mainly used to enlarge an image and increase the resolution of the image. And because the facial features such as spots, moles and pox belong to very small targets, the small targets need to be accurately detected, the network adopts a feature pyramid to generate an image feature map, the feature pyramid fuses the features of adjacent layers to achieve the purpose of feature enhancement, and the detection effect on the small pixel targets is remarkably improved. The feature pyramid network structure is shown in fig. 2, and mainly includes the following processes: bottom up, top down and cross connections. The bottom-up process is a forward propagation process of the neural network, and the feature map is calculated by a convolution kernel and generally becomes smaller and smaller, which is a process of image feature extraction; the top-down process is to up-sample a high-level feature map which is more abstract and has stronger semantics; the cross-linking is to fuse the up-sampled feature map and the feature map generated from bottom to top. It should be noted that the two layers of features that are connected in the transverse direction are the same size in space, so that the detailed information can be located by using the features of the lower layer.
Optionally, the extracting the key feature of the facial diagnosis by using the feature pyramid network structure is shown in fig. 3, and includes:
step 141: extracting features from bottom to top through the feature pyramid to generate a first key feature map;
step 142: after the feature map before the first key feature map is generated is subjected to up-sampling, the feature map is fused with the feature map subjected to feature extraction from the bottom to the top of the feature pyramid, and a second key feature map is generated after convolution kernel calculation;
step 143: after the feature map fused in the previous step is up-sampled, the feature map is fused with the feature map extracted from the bottom to the top by the feature pyramid, and a third key feature map is generated after convolution kernel calculation;
step 144: and the generated key feature graph is combined with the human face feature detection training model to output the corresponding facial diagnosis key feature.
Optionally, the step 143 is additionally executed by the feature pyramid network structure according to the need of extracting the key features of the interview.
Specifically, a network structure of a face feature detection model according to an embodiment of the present disclosure is shown in fig. 4. The input image is subjected to multilayer convolution Conv, Pooling Pooling and feature fusion Add, namely, a process of a feature pyramid from bottom to top is carried out, a feature map F1, namely a first key feature map, is firstly generated, F1 has rich semantic information and has a large receptive field, so that the method is suitable for detecting human face features of a large point, such as spots, transverse striations and the like, the feature map before F1 is subjected to upsampling and is fused with a corresponding feature map extracted from bottom to top to generate a feature map F2, namely a second key feature map, the feature map F3, namely a third key feature map, is generated in the same way, and according to the requirement of the traditional Chinese medicine face diagnosis precision, the feature map F4, namely a fourth key feature map … … can be generated in the same way. It can be seen that feature maps such as F2, F3, and F4 have much spatial position information and a small receptive field, and thus are suitable for detecting relatively small human face features. Because high-order feature semantic information is rich, low-order feature semantic information is less but space information is more, the high-order feature is up-sampled and added and fused with the low-order feature, so that the high-order semantic information can be added to the low-order feature map while the space information is kept, namely, the feature maps such as F2, F3 and F4 are more suitable for detecting small targets such as nevi and pox through the top-down process of the feature pyramid, and the accuracy of small target detection can be obviously improved.
Step 105: and (4) combining the key characteristics of the facial diagnosis and the traditional Chinese medicine facial diagnosis subareas, and obtaining a traditional Chinese medicine facial diagnosis result according to medical logic judgment.
It should be noted that the facial diagnosis key feature is a certain or several facial diagnosis key features in a partition of the face image, that is, in a certain facial partition. According to the viscera regions corresponding to the facial partitions, facial feature identification information such as spots, moles, acne, blood streak and the like is matched, according to the theory of traditional Chinese medicine, relevant ancient books and modern research are analyzed, and the viscera functional state and pathological state reflected by the information such as the quantity, size, morphology and the like of the facial features are mined. Part of the face feature information corresponds to relevant Chinese medicine symptoms, can be directly used for auxiliary judgment,
if nevus exists in the lung area, the risk of lung function related diseases such as pharyngolaryngitis, tonsillitis and the like is increased; detecting characteristic wrinkles in the heart area, and increasing the risk of suffering from heart functional diseases; blood streak, or pox, or plaque, is detected in the renal area, with an increased risk of renal dysfunction. According to different weights and combination modes of the human face features, the visceral function status index is calculated to form a combination of different visceral functions, and the risk of related diseases is judged, for example: the liver area identifies spots (5, 1, 2), 'nevus (1, 2, 1),' pox '(0, 0), the spleen area identifies spots (0, 0, 0),' nevus (0, 0), 'pox' (1, 1, 1), and the auxiliary facial diagnosis result is liver depression and spleen deficiency, wherein in (5, 1, 2), the first number 5 represents the number of the characteristics, and the larger the number, the larger the number of the characteristics; the second number, 1, indicates the size of the feature, with larger numbers indicating larger areas of the feature; the third numeral 2 indicates the shape of the feature, which may be circular, dotted, oval, sheet-like, etc., and is indicated by 1,2,3,4, respectively.
The facial features of the face, such as the nevus and pox, the blood streak, the striation and the like, are the basis for carrying out the logic judgment of the traditional Chinese medicine, and play a role in assisting in improving the accuracy of the diagnosis result of the traditional Chinese medicine facial diagnosis. When the feature pyramid is used for generating 3 feature maps for face feature detection, the detection effect on moles is poor, some moles are not detected, and the facial diagnosis result cannot truly reflect the actual situation of a patient, mainly because the feature maps generated from top to bottom of the feature pyramid contain less semantic information, the detection effect on small targets such as moles is poor. When 5 characteristic graphs are generated by the characteristic pyramid to detect the human face characteristics, the effect of detecting the human face characteristics is equivalent to that of 4 characteristic graphs generated by the characteristic pyramid, but when 5 characteristic graphs are used to detect the human face characteristics, the calculation amount and the occupied calculation resources are far higher than that of 4 characteristic graphs, and the 5 characteristic graphs not only consume more resources, but also have lower calculation speed, so that the 4 characteristic graphs generated by the characteristic pyramid are used to detect the human face characteristics, and the structure can accurately detect the human face characteristics such as spots, nevi, acne, blood streak, transverse striations and the like, so that the diagnosis result of the facial diagnosis is more accurate, the calculation resources can be saved, and the calculation speed is improved.
The model output not only comprises the position information of the face features, but also comprises the category information of the face features, the categories of the detected spots comprise chloasma, freckles, senile plaques and the like, and the categories of the pox comprise acne, pustule, nodules and the like. The sizes and the colors of the facial features such as the spots, the moles and the pox can be obtained through the position information of the facial features such as the spots, the nevus and the pox, the position information of the facial features is represented by a square frame, the square frame comprises two coordinates, the upper left corner coordinate of the square frame and the lower right corner coordinate of the square frame, the facial features such as the spots, the nevus and the pox are approximate to be an ellipse, the length and the width of the square frame can be obtained according to the coordinate information of the square frame, and the area of the minimum inscribed ellipse of the square frame can be obtained according to the length and the width, so that the sizes of the facial features such as the spots, the nevus and the pox can be approximate. Compared with the RGB color space, the HSV color space has more intuitive connection with colors and can conveniently obtain the color information of the image area. The more accurate the position information of the face features, the more accurate the size and color of the obtained face features. The detection result of generating 4 feature maps by the feature pyramid not only can accurately distinguish the types of the human face features, but also can accurately position the human face features. The diagnosis result of the traditional Chinese medicine facial diagnosis is more accurate by combining the information of the category, the size, the color and the like of the human face characteristics and through the traditional Chinese medicine facial diagnosis logic.
Example two
Fig. 5 is a schematic structural diagram of a traditional Chinese medicine facial diagnosis auxiliary diagnosis device 500 based on face feature detection according to an embodiment of the present disclosure. Referring to fig. 5, in one embodiment, the apparatus for assisting diagnosis of facial diagnosis in traditional Chinese medicine based on human face feature detection includes:
the face image acquisition module 501 is configured to acquire a face image.
The model training module 502 is configured to train a face feature detection model according to the face diagnosis key features labeled on the face image, where the face diagnosis key features include at least one of a speckle, a mole, a pox, a blood streak, and a horizontal streak.
The face image partitioning module 503 is configured to obtain a face region required for facial diagnosis by using a face key point detection algorithm, and perform traditional Chinese medicine facial diagnosis partitioning on the face image according to the face key point positioning.
A face key feature extraction module 504, configured to extract the face diagnosis key features from the face image through a convolutional neural network structure, where the convolutional neural network structure includes at least one convolutional layer, one pooling layer, one feature fusion, and one feature pyramid.
And the facial diagnosis result output module 505 is used for obtaining the traditional Chinese medicine facial diagnosis result according to medical logic judgment by combining the facial diagnosis key characteristics and the traditional Chinese medicine facial diagnosis subareas.
It should be understood that the facial feature detection-based traditional Chinese medicine facial diagnosis auxiliary diagnosis device (or apparatus) in the embodiment of the present specification may also execute the method executed by the facial feature detection-based traditional Chinese medicine facial diagnosis auxiliary diagnosis device (or apparatus) in fig. 1 to 4, and implement the functions of the facial feature detection-based traditional Chinese medicine facial diagnosis auxiliary diagnosis device (or apparatus) in the examples shown in fig. 1 to 4, which are not described herein again.
EXAMPLE III
Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present specification. Referring to fig. 6, at a hardware level, the electronic device includes a processor, and optionally further includes an internal bus, a network interface, and a memory. The Memory may include a Memory, such as a Random-Access Memory (RAM), and may further include a non-volatile Memory, such as at least 1 disk Memory. Of course, the electronic device may also include hardware required for other services.
The processor, the network interface, and the memory may be connected to each other via an internal bus, which may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 6, but that does not indicate only one bus or one type of bus.
And the memory is used for storing programs. In particular, the program may include program code comprising computer operating instructions. The memory may include both memory and non-volatile storage and provides instructions and data to the processor.
The processor reads the corresponding computer program from the nonvolatile memory into the memory and then runs the computer program to form the shared resource access control device on the logic level. The processor is used for executing the program stored in the memory and is specifically used for executing the following operations:
collecting a face image;
marking the key facial features of the face image for training a face feature detection model, wherein the key facial features comprise at least one of spots, moles, acne, blood streak and transverse striations;
acquiring a face region required by facial diagnosis by adopting a face key point detection algorithm, and carrying out traditional Chinese medicine facial diagnosis partitioning on the face image according to the face key point positioning;
extracting the key features of the facial diagnosis from the face image through a convolutional neural network structure, wherein the convolutional neural network structure comprises at least one convolutional layer, one pooling layer, one feature fusion and one feature pyramid;
and (4) combining the key characteristics of the facial diagnosis and the traditional Chinese medicine facial diagnosis subareas, and obtaining a traditional Chinese medicine facial diagnosis result according to medical logic judgment.
The above-mentioned facial feature detection-based traditional Chinese medicine diagnosis assisting method disclosed in the embodiments shown in fig. 1 to fig. 4 of the present specification can be applied to a processor, or implemented by the processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components. The various methods, steps and logic blocks disclosed in the embodiments of the present specification may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present specification may be embodied directly in a hardware decoding processor, or in a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor.
Of course, besides the software implementation, the electronic device of the embodiment of the present disclosure does not exclude other implementations, such as a logic device or a combination of software and hardware, and the like, that is, the execution subject of the following processing flow is not limited to each logic unit, and may also be hardware or a logic device.
Example four
Embodiments of the present specification also propose a computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a portable electronic device comprising a plurality of application programs, are capable of causing the portable electronic device to perform the method of the embodiments shown in fig. 1 to 4, and in particular to perform the method of:
collecting a face image;
marking the key facial features of the face image for training a face feature detection model, wherein the key facial features comprise at least one of spots, moles, acne, blood streak and transverse striations;
acquiring a face region required by facial diagnosis by adopting a face key point detection algorithm, and carrying out traditional Chinese medicine facial diagnosis partitioning on the face image according to the face key point positioning;
extracting the key features of the facial diagnosis from the face image through a convolutional neural network structure, wherein the convolutional neural network structure comprises at least one convolutional layer, one pooling layer, one feature fusion and one feature pyramid;
and (4) combining the key characteristics of the facial diagnosis and the traditional Chinese medicine facial diagnosis subareas, and obtaining a traditional Chinese medicine facial diagnosis result according to medical logic judgment.
In short, the above description is only a preferred embodiment of the present disclosure, and is not intended to limit the scope of the present disclosure. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present specification shall be included in the protection scope of the present specification.
The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. One typical implementation device is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
Claims (12)
1. A traditional Chinese medicine face diagnosis auxiliary diagnosis method based on face feature detection comprises the steps of collecting face images, and is characterized by further comprising the following steps:
marking the key facial features of the face image for training a face feature detection model, wherein the key facial features comprise at least one of spots, moles, acne, blood streak and transverse striations;
acquiring a face region required by facial diagnosis by adopting a face key point detection algorithm, and carrying out traditional Chinese medicine facial diagnosis partitioning on the face image according to the face key point positioning;
extracting the key features of the facial diagnosis from the face image through a convolutional neural network structure, wherein the convolutional neural network structure comprises at least one convolutional layer, one pooling layer, one feature fusion and one feature pyramid;
and (4) combining the key characteristics of the facial diagnosis and the traditional Chinese medicine facial diagnosis subareas, and obtaining a traditional Chinese medicine facial diagnosis result according to medical logic judgment.
2. The method of claim 1, wherein the face keypoint detection algorithm is a face keypoint detection algorithm carried by a dlib library.
3. The method of claim 1, wherein said areas of facial diagnosis in TCM comprise at least the cardiac, pulmonary, hepatic, splenic, biliary, and renal areas.
4. The method of claim 1, wherein the feature pyramid network structure extracting the critical facial features comprises:
step 141: extracting features from bottom to top through the feature pyramid to generate a first key feature map;
step 142: after the feature map before the first key feature map is generated is subjected to up-sampling, the feature map is fused with the feature map subjected to feature extraction from the bottom to the top of the feature pyramid, and a second key feature map is generated after convolution kernel calculation;
step 143: after the feature map fused in the previous step is up-sampled, the feature map is fused with the feature map extracted from the bottom to the top by the feature pyramid, and a third key feature map is generated after convolution kernel calculation;
step 144: and the generated key feature graph is combined with the human face feature detection training model to output the corresponding facial diagnosis key feature.
5. The method of claim 4, wherein the feature pyramid network structure performs step 143 as needed to extract the critical facial features.
6. A traditional Chinese medicine face diagnosis auxiliary diagnosis device based on face feature detection is characterized by comprising:
the face image acquisition module is used for acquiring a face image;
the model training module is used for training a face feature detection model according to face diagnosis key features marked on the face image, wherein the face diagnosis key features comprise at least one of spots, moles, acne, blood streak and horizontal lines;
the face image partitioning module is used for acquiring a face region required by face diagnosis by adopting a face key point detection algorithm so as to perform traditional Chinese medicine face diagnosis partitioning on the face image according to the face key point positioning;
the face key feature extraction module is used for extracting the face diagnosis key features from the face image through a convolutional neural network structure, wherein the convolutional neural network structure comprises at least one convolution layer, one pooling layer, one feature fusion and one feature pyramid;
and the facial diagnosis result output module is used for combining the facial diagnosis key characteristics and the traditional Chinese medicine facial diagnosis subareas and obtaining a traditional Chinese medicine facial diagnosis result according to medical logic judgment.
7. The apparatus of claim 6, wherein the face key point detection algorithm of the face image partition module is a face key point detection algorithm of a dlib library.
8. The apparatus of claim 6, wherein said TCM facial diagnosis area includes at least one of a heart area, a lung area, a liver area, a spleen area, a gallbladder area, and a kidney area.
9. The apparatus of claim 6, wherein the feature pyramid network structure of the face key feature extraction module extracting the face key features comprises:
step 141: extracting features from bottom to top through the feature pyramid to generate a first key feature map;
step 142: after the feature map before the first key feature map is generated is subjected to up-sampling, the feature map is fused with the feature map subjected to feature extraction from the bottom to the top of the feature pyramid, and a second key feature map is generated after convolution kernel calculation;
step 143: after the feature map fused in the previous step is up-sampled, the feature map is fused with the feature map extracted from the bottom to the top by the feature pyramid, and a third key feature map is generated after convolution kernel calculation;
step 144: and the generated key feature graph is combined with the human face feature detection training model to output the corresponding facial diagnosis key feature.
10. The apparatus of claim 9, wherein the feature pyramid network structure of the face key feature extraction module performs step 143 additionally according to the need of extracting the face key features.
11. An electronic device, comprising:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to perform the method of any of claims 1 to 5.
12. A computer readable storage medium, characterized in that the computer readable storage medium stores one or more programs that, when executed by an electronic device comprising a plurality of application programs, cause the electronic device to perform the method of any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210093004.3A CN114121269B (en) | 2022-01-26 | 2022-01-26 | Traditional Chinese medicine facial diagnosis auxiliary diagnosis method and device based on face feature detection and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210093004.3A CN114121269B (en) | 2022-01-26 | 2022-01-26 | Traditional Chinese medicine facial diagnosis auxiliary diagnosis method and device based on face feature detection and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114121269A true CN114121269A (en) | 2022-03-01 |
| CN114121269B CN114121269B (en) | 2022-07-15 |
Family
ID=80361538
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210093004.3A Active CN114121269B (en) | 2022-01-26 | 2022-01-26 | Traditional Chinese medicine facial diagnosis auxiliary diagnosis method and device based on face feature detection and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114121269B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115017331A (en) * | 2022-06-07 | 2022-09-06 | 深圳伯德睿捷健康科技有限公司 | Health perception and cognition method and device |
| CN115830009A (en) * | 2023-02-07 | 2023-03-21 | 智慧眼科技股份有限公司 | Diagnosis-by-diagnosis system, method, device and medium based on expert knowledge map reasoning |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106778047A (en) * | 2017-03-06 | 2017-05-31 | 武汉嫦娥医学抗衰机器人股份有限公司 | A kind of traditional Chinese medical science facial diagnosis integrated system based on various dimensions medical image |
| CN108846327A (en) * | 2018-05-29 | 2018-11-20 | 中国人民解放军总医院 | A kind of intelligent distinguishing system and method for mole and melanoma |
| CN109711306A (en) * | 2018-12-19 | 2019-05-03 | 新绎健康科技有限公司 | A kind of method and apparatus obtaining facial characteristics based on depth convolutional neural networks |
| CN112200161A (en) * | 2020-12-03 | 2021-01-08 | 北京电信易通信息技术股份有限公司 | Face recognition detection method based on mixed attention mechanism |
| CN112464885A (en) * | 2020-12-14 | 2021-03-09 | 上海交通大学 | Image processing system for future change of facial color spots based on machine learning |
| CN113744178A (en) * | 2020-08-06 | 2021-12-03 | 西北师范大学 | Skin lesion segmentation method based on convolution attention model |
| US20210406523A1 (en) * | 2019-08-19 | 2021-12-30 | Shenzhen Sensetime Technology Co., Ltd. | Method and device for detecting living body, electronic device and storage medium |
-
2022
- 2022-01-26 CN CN202210093004.3A patent/CN114121269B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106778047A (en) * | 2017-03-06 | 2017-05-31 | 武汉嫦娥医学抗衰机器人股份有限公司 | A kind of traditional Chinese medical science facial diagnosis integrated system based on various dimensions medical image |
| CN108846327A (en) * | 2018-05-29 | 2018-11-20 | 中国人民解放军总医院 | A kind of intelligent distinguishing system and method for mole and melanoma |
| CN109711306A (en) * | 2018-12-19 | 2019-05-03 | 新绎健康科技有限公司 | A kind of method and apparatus obtaining facial characteristics based on depth convolutional neural networks |
| US20210406523A1 (en) * | 2019-08-19 | 2021-12-30 | Shenzhen Sensetime Technology Co., Ltd. | Method and device for detecting living body, electronic device and storage medium |
| CN113744178A (en) * | 2020-08-06 | 2021-12-03 | 西北师范大学 | Skin lesion segmentation method based on convolution attention model |
| CN112200161A (en) * | 2020-12-03 | 2021-01-08 | 北京电信易通信息技术股份有限公司 | Face recognition detection method based on mixed attention mechanism |
| CN112464885A (en) * | 2020-12-14 | 2021-03-09 | 上海交通大学 | Image processing system for future change of facial color spots based on machine learning |
Non-Patent Citations (4)
| Title |
|---|
| ZHIYUAN LI ET AL.: "POOLING MAP ADAPTATION IN CONVOLUTIONAL NEURAL NETWORK FOR FACIAL EXPRESSION RECOGNITION", 《2019 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO》 * |
| 张业 等: "基于SR-VGG19的人脸表情识别算法研究", 《计算机与数字工程》 * |
| 王松: "中医面诊客观化中若干图像分析技术研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 陈伟 等: "基于深度卷积神经网络的黑色素瘤诊断算法及临床验证", 《组织工程与重建外科》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115017331A (en) * | 2022-06-07 | 2022-09-06 | 深圳伯德睿捷健康科技有限公司 | Health perception and cognition method and device |
| CN115830009A (en) * | 2023-02-07 | 2023-03-21 | 智慧眼科技股份有限公司 | Diagnosis-by-diagnosis system, method, device and medium based on expert knowledge map reasoning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114121269B (en) | 2022-07-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| ES2731327T3 (en) | Method to characterize images acquired through a medical video device | |
| CN114121269B (en) | Traditional Chinese medicine facial diagnosis auxiliary diagnosis method and device based on face feature detection and storage medium | |
| Wang et al. | Smartphone-based wound assessment system for patients with diabetes | |
| Alqahtani et al. | Breast cancer pathological image classification based on the multiscale CNN squeeze model | |
| An et al. | Medical image segmentation algorithm based on multilayer boundary perception-self attention deep learning model | |
| Sparks et al. | Out-of-sample extrapolation utilizing semi-supervised manifold learning (OSE-SSL): content based image retrieval for histopathology images | |
| CN113920309B (en) | Image detection method, image detection device, medical image processing equipment and storage medium | |
| Yu et al. | Accurate system for automatic pill recognition using imprint information | |
| Hu et al. | Content‐based gastric image retrieval using convolutional neural networks | |
| US20210097377A1 (en) | Method and apparatus for image recognition | |
| Yu et al. | Traffic sign detection based on visual co-saliency in complex scenes | |
| CN113888541A (en) | Image identification method, device and storage medium for laparoscopic surgery stage | |
| Li et al. | Staged detection–identification framework for cell nuclei in histopathology images | |
| Azaza et al. | Context proposals for saliency detection | |
| Lang et al. | Dual low-rank pursuit: Learning salient features for saliency detection | |
| Zeng et al. | Efficient fetal ultrasound image segmentation for automatic head circumference measurement using a lightweight deep convolutional neural network | |
| Liu et al. | MFBGR: Multi-scale feature boundary graph reasoning network for polyp segmentation | |
| Liu et al. | Automatic segmentation and measurement of pressure injuries using deep learning models and a LiDAR camera | |
| Yuan et al. | An improved yolov5 for skull fracture detection | |
| Liu et al. | HMA‐Net: A deep U‐shaped network combined with HarDNet and multi‐attention mechanism for medical image segmentation | |
| CN114446469A (en) | Face region segmentation method, device and storage medium for traditional Chinese medicine face diagnosis auxiliary diagnosis | |
| CN112785601B (en) | Image segmentation method, system, medium and electronic terminal | |
| Yan et al. | Multilevel feature aggregation and enhancement network for remote sensing change detection | |
| Oh et al. | Detection of multiple salient objects by categorizing regional features | |
| CN113657245A (en) | Method, device, medium and program product for human face living body detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |