WO2022041396A1 - Ocular surface features-based novel coronavirus pneumonia (covid-19) disease patient risk screening deep learning system - Google Patents

Ocular surface features-based novel coronavirus pneumonia (covid-19) disease patient risk screening deep learning system Download PDF

Info

Publication number
WO2022041396A1
WO2022041396A1 PCT/CN2020/119241 CN2020119241W WO2022041396A1 WO 2022041396 A1 WO2022041396 A1 WO 2022041396A1 CN 2020119241 W CN2020119241 W CN 2020119241W WO 2022041396 A1 WO2022041396 A1 WO 2022041396A1
Authority
WO
WIPO (PCT)
Prior art keywords
picture
eye
disease
face
covid
Prior art date
Application number
PCT/CN2020/119241
Other languages
French (fr)
Chinese (zh)
Inventor
付彦伟
薛向阳
顾梦炜
Original Assignee
复旦大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 复旦大学 filed Critical 复旦大学
Publication of WO2022041396A1 publication Critical patent/WO2022041396A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7271—Specific aspects of physiological measurement analysis
    • A61B5/7275—Determining trends in physiological measurement data; Predicting development of a medical condition based on physiological measurements, e.g. determining a risk factor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems

Definitions

  • the invention belongs to the cross field of medical image and computer image recognition, and in particular relates to a deep learning system for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface features.
  • COVID-19 new coronary pneumonia
  • DL-based artificial intelligence techniques have made remarkable progress in various computer vision tasks such as object detection, image classification, instance segmentation, and object recognition.
  • the present invention is made to solve the above problems, and aims to provide a deep learning system for risk screening of disease patients based on ocular surface characteristics.
  • the present invention provides a deep learning system for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface features, which is used to screen the disease risk of patients by using the eye area in a photographed face picture, It has such features, including: a face image preprocessing part, which is used to preprocess the face image and obtain an eye area image; an eye image feature extraction part, which is used for extracting through the trained eye image feature extraction model.
  • the basic features of eye area pictures; the classification part is used to predict the disease category of the new coronary pneumonia (COVID-19) at the picture level according to the basic features and obtain the picture-level classification results, as well as the patient-level classification results based on the picture-level classification results.
  • the patient-level prediction results are obtained by classification, and the training of the eye image feature extraction model includes the following steps:
  • Step S1 input the eye region picture into the eye image feature extraction model to extract basic features, and the classification unit predicts whether the corresponding patient in the eye region picture suffers from new coronary pneumonia (COVID-19) according to the basic features and outputs the picture-level classification result; step S2, by constructing a loss function, supervise the basic features extracted by the eye image feature extraction model according to the picture-level classification results and the actual disease category of the patient corresponding to the eye area image; step S3, using the SGD optimizer to analyze the eye image
  • the feature extraction model is iteratively trained.
  • the face image preprocessing part also has a face detection model
  • the preprocessing of the face image by the face image preprocessing part includes the following steps: step E1, obtaining the position area of the face in the face image and the coordinates of the facial key points through the face detection model; step E2, according to the facial key point coordinates in the The key points of the left and right eyes of the face are obtained, and the maximum and minimum values of the horizontal and vertical coordinates of the face eye area are obtained.
  • the horizontal and vertical coordinates are expanded outward by a certain value to ensure that all the face and eye areas are included in the feature extraction range.
  • the eye intercepted pictures are The pictures with the longitudinal length longer than the horizontal length are removed to obtain the pictures of the eye region.
  • the deep learning system for risk screening of disease patients based on ocular surface features may also have the following feature: when the classification unit performs patient-level classification, the most urgent disease in the disease category is set as the highest Priority disease category, for a patient's multiple eye region pictures, when the number of the highest priority disease category in the predicted picture-level classification results is greater than or equal to 1, it is determined that the patient has the disease. It is more likely to obtain patient-level prediction results.
  • the deep learning system for risk screening of new coronary pneumonia (COVID-19) disease patients based on ocular surface characteristics involved in the present invention because the ocular surface of the disease patients often has conjunctivitis-like manifestations, including conjunctival congestion, stasis removal, overflow
  • the basic features extracted by the eye image feature extraction model are used to predict the disease category at the picture level and the patient level.
  • the learning and expression of features is carried out in the region, and the risk screening of disease patients based on eye features can be realized by capturing more distinguishing and discriminating features; Pictures can be used to screen patients with diseases, which can improve the speed, accuracy and convenience of disease risk screening. At the same time, it can get rid of restrictions such as dependence on professionals. Detection, dynamic monitoring of the degree of virus infection, observation of treatment effects, epidemic tracking and epidemic mapping, so as to achieve efficient epidemic prevention and control.
  • FIG. 1 is a structural block diagram of a deep learning system for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface features in an embodiment of the present invention
  • FIG. 2 is a system diagram of a deep learning system for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface characteristics in an embodiment of the present invention
  • FIG. 3 is a flowchart of face picture preprocessing performed by the deep learning system for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface features in an embodiment of the present invention
  • FIG. 4 is a flowchart of training an eye image feature extraction model in the deep learning system for risk screening of new coronary pneumonia (COVID-19) disease patients based on ocular surface features in an embodiment of the present invention
  • FIG. 5 is a schematic diagram of the disease screening process of the deep learning system for risk screening of new coronary pneumonia (COVID-19) disease patients based on ocular surface characteristics in an embodiment of the present invention
  • FIG. 6 is a block diagram of a disease screening process of the deep learning system for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface characteristics in an embodiment of the present invention.
  • FIG. 1 is a structural block diagram of a deep learning system for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface characteristics in an embodiment of the present invention.
  • COVID-19 new coronary pneumonia
  • the deep learning system 100 for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface features is used to screen the disease risk of patients by taking the eye area in the obtained face picture , including a face image preprocessing part 1, an eye image feature extraction part 2, a classification part 3, a disease risk screening evaluation part 4, an output display part 5, a system communication part 6 and a system control part 7 for controlling the above parts .
  • the deep learning system 100 for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface characteristics is composed of a computer device 110 and a result display device 140, and is used for external media data. to be processed.
  • the computer device is composed of a processor 120 and a memory 130: the processor 120 is a hardware processor for computing and running executable codes, such as a central processing unit CPU or a graphics computing processor GPU; the memory 130 is a non-volatile
  • the storage device is used to store executable codes so as to allow the processor 120 to perform corresponding computing processes. Meanwhile, the memory 130 also stores various intermediate data and parameters.
  • the storage content of the memory 130 includes a face data set storing face pictures and related parameters, executable codes, and the executable codes are used to run the face image preprocessing part 1 , the eye image feature extraction part 2 and the classification part 3 .
  • the media data is captured by various data collection devices, such as a smart phone, and the media data may be video content or image content.
  • the media data may also be face image data including multiple different identities and disease types, and the face pictures are obtained from the media data.
  • the face image preprocessing part 1 is used to preprocess the face image and obtain the eye region image.
  • the face image preprocessing part 1 has a face detection model, and the face image preprocessing part 1 preprocesses the face image based on the face detection model, as shown in Figure 3, the preprocessing comprises the following steps:
  • step S1 the position area of the face in the face picture and the coordinates of the key points of the face are obtained through the face detection model.
  • the original face picture usually includes not only the eye area, but also the background or other areas of the face, such as the nose, ears, and mouth. If the original face image is directly used for classification, it will inevitably introduce noise or irrelevant information, such as background noise or nose features, making the performance of the eye image feature extraction model inaccurate and unreliable. Therefore, in order to make the eye image feature extraction model focus on extracting eye surface features, the present embodiment adopts the face detection model to obtain the position area of the face in the picture and the coordinates of the facial key points, and the facial key point coordinates include the left and right eyes of the human face. And the position coordinates of other organs of the face.
  • Step S2 according to the key points of the left and right eyes of the face in the key point coordinates of the face, obtain the maximum value and the minimum value of the horizontal and vertical coordinates of the eye region of the human face, and at the same time, the horizontal and vertical coordinates are expanded outward by a certain value to ensure the face.
  • the eye area is all included in the feature extraction range, and the eye area of the face is intercepted to obtain an eye interception image.
  • Step S3 Screen the eye intercepted pictures to obtain eye region pictures.
  • the pictures of the eye clipped pictures whose vertical length is longer than the horizontal length are eliminated to obtain the eye region pictures.
  • the facial key point coordinates obtained by the face detection model are not completely accurate, so the interception of the face eye region will also have corresponding deviations.
  • the eye area of the human face should be a rectangular area with a longer horizontal length, any vertical length longer than horizontal in the aspect ratio is considered to be the result of the failure to locate the eye area of the human face, so such eyes need to be intercepted. Image culling.
  • the eye image feature extraction part 2 is used for extracting the basic features of the eye region picture through the trained eye image feature extraction model.
  • FIG. 4 is a flowchart of training an eye image feature extraction model in the deep learning system for risk screening of patients with novel coronavirus pneumonia (COVID-19) disease based on ocular surface features in an embodiment of the present invention.
  • COVID-19 novel coronavirus pneumonia
  • the training of the eye image feature extraction model includes the following steps:
  • Step E1 input the eye area image into the eye image feature extraction model to extract basic features, and the classification unit predicts the new coronary pneumonia (COVID-19) disease category of the corresponding patient in the eye area image according to the basic features and outputs the image-level classification result .
  • COVID-19 new coronary pneumonia
  • Step E2 by constructing a loss function, supervise the basic features extracted by the eye image feature extraction model according to the picture-level classification result and the actual disease category of the patient corresponding to the eye region picture.
  • Step E3 using the SGD optimizer to iteratively train the eye image feature extraction model.
  • the classification part 3 is used to perform picture-level prediction of the disease category of the novel coronavirus pneumonia (COVID-19) according to the basic features and obtain picture-level classification results, and to perform patient-level classification according to the picture-level classification results to obtain patient-level prediction results.
  • COVID-19 novel coronavirus pneumonia
  • the picture-level disease category prediction is to determine whether the patient corresponding to the eye region picture has a disease or other disease (for example, common pneumonia or eye disease, etc.).
  • a disease or other disease for example, common pneumonia or eye disease, etc.
  • the classification unit 3 When the classification unit 3 performs patient-level classification, the most urgent disease in the disease category is set as the highest-priority disease category. For a patient's multiple eye region pictures, when the predicted picture-level classification result When the number of the highest priority disease category is greater than or equal to 1, it is judged that the patient is more likely to suffer from the disease, and a patient-level prediction result is obtained.
  • the disease risk screening and evaluation unit 4 is configured to perform disease risk evaluation according to the picture-level classification result and the patient-level prediction result.
  • the result display device includes an output display unit 5 for playing media data and displaying picture-level classification results and patient-level prediction results, which can be a computer, a TV, or a mobile device.
  • FIG. 5 is a schematic diagram of a disease screening process flow of the deep learning system for risk screening of disease patients based on ocular surface features in an embodiment of the present invention
  • FIG. 6 is an embodiment of the present invention. 19) A block diagram of the disease screening process of the deep learning system for risk screening of disease patients.
  • the disease screening process of the deep learning system for risk screening of new coronary pneumonia (COVID-19) disease patients based on ocular surface characteristics of this embodiment includes the following steps:
  • Step T1 the face image preprocessing part 1 preprocesses the captured face image to obtain an eye region image, and then enters step T2;
  • Step T2 the eye image feature extraction part 2 extracts the basic features of the eye region picture according to the eye region picture, and then enters step T3;
  • Step T3 the classification unit 3 performs picture-level disease category prediction according to the basic features, predicts whether the patient has a disease or other disease (such as common pneumonia or eye disease, etc.) corresponding to the picture, obtains the picture-level classification result, and then enters step T4 ;
  • a disease or other disease such as common pneumonia or eye disease, etc.
  • Step T4 the classification unit 3 sets the most urgent disease in the disease category as the highest priority disease class, and when the number of the highest priority disease class in the picture-level classification result is greater than or equal to 1, it is determined that the patient suffers from the disease. The possibility of the disease is high, the patient-level prediction result is obtained, and then the step T5 is entered;
  • step T5 the output display unit 5 displays the picture-level classification result and the patient-level prediction result for the user to view, and then enters the end state.
  • the deep learning system for risk screening of disease patients based on ocular surface characteristics involved in this embodiment, because the ocular surface of disease patients often has conjunctivitis-like manifestations, including conjunctival congestion, stasis removal, discharge or increased secretion, etc.
  • the basic features extracted by the eye image feature extraction model are used to predict the disease category at the picture level and the patient level, and the learning and expression of features for the face and eye area , by capturing more distinguishing and discriminating features, the risk screening of disease patients based on eye features can be realized; and the present invention can carry out disease patients by taking a face picture and according to the eye area picture in the face picture.
  • the screening work can improve the speed, accuracy and convenience of disease risk screening, and at the same time, it can get rid of restrictions such as professional reliance, and can be popularized on a large scale.
  • quantitative detection can be realized anytime and anywhere, and virus infection can be dynamically monitored. To achieve efficient epidemic prevention and control.
  • the image of the eye region is obtained by preprocessing and screening the face image first, and then inputting the eye image feature extraction model, it can be ensured that the input pictures are all the images of the eye region with accurate positioning, and the face image is effectively removed.
  • the noise and irrelevant information in the eye image ensure that the eye image feature extraction model can correctly process the eye region picture.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Medical Informatics (AREA)
  • Biomedical Technology (AREA)
  • General Physics & Mathematics (AREA)
  • Public Health (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Pathology (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • Evolutionary Computation (AREA)
  • Multimedia (AREA)
  • Biophysics (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Heart & Thoracic Surgery (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Human Computer Interaction (AREA)
  • Primary Health Care (AREA)
  • Veterinary Medicine (AREA)
  • Epidemiology (AREA)
  • Evolutionary Biology (AREA)
  • Psychiatry (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Ophthalmology & Optometry (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Quality & Reliability (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Physiology (AREA)
  • Signal Processing (AREA)

Abstract

An ocular surface features-based novel coronavirus pneumonia (COVID-19) disease patient risk screening deep learning system, relating to the cross field of medical images and computer image recognition, and being used for performing COVID-19 disease risk screening of a patient by means of the ocular region in a face picture obtained by means of photographing. The deep learning system comprises: a face image pre-processing unit (1), for pre-processing the face picture and obtaining an ocular region picture; an ocular image feature extraction unit (2), for extracting basic features of the ocular region picture by means of a trained ocular image feature extraction model; and a picture-level and patient-level classification module unit (3), for performing, according to the basic features, picture-level disease category prediction and obtaining a picture-level classification result, and for performing patient-level classification according to the picture-level classification result and obtaining a COVID-19 patient-level prediction result.

Description

一种基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习***A deep learning system for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface features 技术领域technical field
本发明属于医疗影像和计算机图像识别交叉领域，具体涉及一种基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习***。The invention belongs to the cross field of medical image and computer image recognition, and in particular relates to a deep learning system for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface features.
背景技术Background technique
在过去的几十年里，基于深度学习(DL)的人工智能技术在各种计算机视觉任务如目标检测、图像分类、实例分割和物体识别等方面取得了显著的进展。Over the past few decades, deep learning (DL)-based artificial intelligence techniques have made remarkable progress in various computer vision tasks such as object detection, image classification, instance segmentation, and object recognition.
近年来，深度学习的优势使其在医学图像分析中也得到了广泛的应用，例如，对不同疾病进行分类，自闭症谱系障碍或大脑中的阿尔茨海默病，乳腺癌，糖尿病性视网膜病变和青光眼，以及肺癌或肺炎等常见病症。In recent years, the advantages of deep learning have also made it widely used in medical image analysis, for example, to classify different diseases, autism spectrum disorder or Alzheimer's disease in the brain, breast cancer, diabetic retina Lesions and glaucoma, as well as common conditions such as lung cancer or pneumonia.
此外，已有一些工作采用深度学习技术学习并提取CT影像特征进行疾病患者的识别与筛查，并取得良好效果。但是现有的疾病患者筛查技术中存在时效性差、设备要求高、依赖专业人员等不足，CT影像的拍摄需要使用专业的CT设备进行拍摄并需要依赖专业人员进行操作，同时由于进行拍摄并成像的耗时较长，所以无法快速对CT影像特征进行提取并完成疾病患者的识别与筛查。In addition, some works have used deep learning technology to learn and extract CT image features for identification and screening of disease patients, and have achieved good results. However, the existing screening technologies for patients with diseases have shortcomings such as poor timeliness, high equipment requirements, and reliance on professionals. The shooting of CT images requires professional CT equipment to shoot and needs to rely on professionals for operation. At the same time, due to shooting and imaging It takes a long time, so it is impossible to quickly extract CT image features and complete the identification and screening of disease patients.
发明内容SUMMARY OF THE INVENTION
本发明是为了解决上述问题而进行的，目的在于提供一种基于眼表特征的疾病患者风险筛查深度学习***。The present invention is made to solve the above problems, and aims to provide a deep learning system for risk screening of disease patients based on ocular surface characteristics.
本发明提供了一种基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习***，用于通过拍摄得到的人脸图片中的眼部区域来进行患者的疾病风险筛查，具有这样的特征，包括：人脸图像预处理部，用于对人脸图片进行预处理并得到眼部区域图片；眼部图像特征提取部，用于通过训练后的眼部图像特征提取模型提取眼部区域图片的基础特征；分类部，用于根据基础特征进行图片级的新冠肺炎(COVID-19)患病类别预测并得到图片级分类结果，以及用于根据图片级分类结果进行病患级分类得到病患级预测结果，其中，眼部图像特征提取模型训练时包括以下步骤：The present invention provides a deep learning system for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface features, which is used to screen the disease risk of patients by using the eye area in a photographed face picture, It has such features, including: a face image preprocessing part, which is used to preprocess the face image and obtain an eye area image; an eye image feature extraction part, which is used for extracting through the trained eye image feature extraction model. The basic features of eye area pictures; the classification part is used to predict the disease category of the new coronary pneumonia (COVID-19) at the picture level according to the basic features and obtain the picture-level classification results, as well as the patient-level classification results based on the picture-level classification results. The patient-level prediction results are obtained by classification, and the training of the eye image feature extraction model includes the following steps:
步骤S1，将眼部区域图片输入眼部图像特征提取模型中提取基础特征，分类部根据基础特征预测眼部区域图片中对应患者是否罹患新冠肺炎(COVID-19)并输出图片级分类结果；步骤S2，通过构建损失函数，根据图片级分类结果和眼部区域图片对应患者的真实患病类别，对眼部图像特征提取模型提取的基础特征进行监督；步骤S3，采用SGD优化器对眼部图像特征提取模型进行迭代训练。Step S1, input the eye region picture into the eye image feature extraction model to extract basic features, and the classification unit predicts whether the corresponding patient in the eye region picture suffers from new coronary pneumonia (COVID-19) according to the basic features and outputs the picture-level classification result; step S2, by constructing a loss function, supervise the basic features extracted by the eye image feature extraction model according to the picture-level classification results and the actual disease category of the patient corresponding to the eye area image; step S3, using the SGD optimizer to analyze the eye image The feature extraction model is iteratively trained.
在本发明提供的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习***中，还可以具有这样的特征：其中，人脸图像预处理部中还具有人脸检测模型，人脸图像预处理部对人脸图片的预处 理包括以下步骤：步骤E1，通过人脸检测模型获取人脸图片中人脸的位置区域以及面部关键点坐标；步骤E2，根据面部关键点坐标中的人脸左右眼关键点，得到人脸眼部区域的横纵坐标的最大值和最小值，同时对横纵坐标进行一定数值的向外扩充来确保人脸眼部区域全部纳入特征提取范围内，并对人脸眼部区域进行截取，得到眼部截取图片；步骤E3，对眼部截取图片进行筛选，得到眼部区域图片。In the deep learning system for risk screening of new coronary pneumonia (COVID-19) disease patients based on ocular surface features provided by the present invention, it may also have the following features: wherein, the face image preprocessing part also has a face detection model, The preprocessing of the face image by the face image preprocessing part includes the following steps: step E1, obtaining the position area of the face in the face image and the coordinates of the facial key points through the face detection model; step E2, according to the facial key point coordinates in the The key points of the left and right eyes of the face are obtained, and the maximum and minimum values of the horizontal and vertical coordinates of the face eye area are obtained. At the same time, the horizontal and vertical coordinates are expanded outward by a certain value to ensure that all the face and eye areas are included in the feature extraction range. , and intercept the eye region of the human face to obtain a picture of the eye region; step E3, screen the intercepted pictures of the eye region to obtain a picture of the eye region.
在本发明提供的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习***中，还可以具有这样的特征：其中，对眼部截取图片进行筛选时，将眼部截取图片中纵向长度长于横向长度的图片进行剔除后得到眼部区域图片。In the deep learning system for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface characteristics provided by the present invention, it may also have the following feature: wherein, when screening the eye intercepted pictures, the eye intercepted pictures are The pictures with the longitudinal length longer than the horizontal length are removed to obtain the pictures of the eye region.
在本发明提供的基于眼表特征的疾病患者风险筛查深度学习***中，还可以具有这样的特征：其中，分类部进行病患级分类时，将患病类别中最紧急的疾病设为最高优先级别患病类别，对于一位患者的多张眼部区域图片中，当预测得到的图片级分类结果中的最高优先级别患病类别的个数大于等于1时，判断患者患有该疾病的可能性较大，得到病患级预测结果。In the deep learning system for risk screening of disease patients based on ocular surface features provided by the present invention, it may also have the following feature: when the classification unit performs patient-level classification, the most urgent disease in the disease category is set as the highest Priority disease category, for a patient's multiple eye region pictures, when the number of the highest priority disease category in the predicted picture-level classification results is greater than or equal to 1, it is determined that the patient has the disease. It is more likely to obtain patient-level prediction results.
发明的作用与效果The role and effect of the invention
根据本发明所涉及的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习***，因为能够根据疾病患者眼表往往有着类结膜炎的表现，包括结膜充血、化瘀、溢液或分泌物增多等特征，借助深度学习网络的特征提取和分类预测能力，通过眼部图像特征提取模 型提取的基础特征来进行图片级以及病患级的患病类别预测，针对人脸眼部区域进行特征的学习表达，通过捕获更具分辨力和识别力的特征，能够实现基于眼部特征的疾病患者风险筛查；并且本发明通过拍摄人脸图片并根据人脸图片中的眼部区域图片就能够进行疾病患者的筛查工作，能够提高疾病风险筛查的快捷性、准确性以及方便性，同时能够摆脱专业人员依赖等限制，可大规模普及，在疫情阶段能够实现随时随地的定量检测，动态监测病毒感染的程度，观察治疗效果，进行疫情跟踪和疫情地图绘制，从而实现高效的疫情防控。According to the deep learning system for risk screening of new coronary pneumonia (COVID-19) disease patients based on ocular surface characteristics involved in the present invention, because the ocular surface of the disease patients often has conjunctivitis-like manifestations, including conjunctival congestion, stasis removal, overflow With the help of the feature extraction and classification prediction capabilities of the deep learning network, the basic features extracted by the eye image feature extraction model are used to predict the disease category at the picture level and the patient level. The learning and expression of features is carried out in the region, and the risk screening of disease patients based on eye features can be realized by capturing more distinguishing and discriminating features; Pictures can be used to screen patients with diseases, which can improve the speed, accuracy and convenience of disease risk screening. At the same time, it can get rid of restrictions such as dependence on professionals. Detection, dynamic monitoring of the degree of virus infection, observation of treatment effects, epidemic tracking and epidemic mapping, so as to achieve efficient epidemic prevention and control.
附图说明Description of drawings
图1是本发明的实施例中的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习***的结构框图；1 is a structural block diagram of a deep learning system for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface features in an embodiment of the present invention;
图2是本发明的实施例中的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习***的***图；2 is a system diagram of a deep learning system for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface characteristics in an embodiment of the present invention;
图3是本发明的实施例中的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习***进行人脸图片预处理的流程图；3 is a flowchart of face picture preprocessing performed by the deep learning system for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface features in an embodiment of the present invention;
图4是本发明的实施例中的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习***中训练眼部图像特征提取模型的流程图；4 is a flowchart of training an eye image feature extraction model in the deep learning system for risk screening of new coronary pneumonia (COVID-19) disease patients based on ocular surface features in an embodiment of the present invention;
图5是本发明的实施例中的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习***的疾病筛查流程示意图；5 is a schematic diagram of the disease screening process of the deep learning system for risk screening of new coronary pneumonia (COVID-19) disease patients based on ocular surface characteristics in an embodiment of the present invention;
图6是本发明的实施例中的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习***的疾病筛查流程框图。6 is a block diagram of a disease screening process of the deep learning system for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface characteristics in an embodiment of the present invention.
具体实施方式detailed description
为了使本发明实现的技术手段与功效易于明白了解，以下结合实施例及附图对本发明作具体阐述。In order to make the technical means and effects realized by the present invention easy to understand, the present invention will be described in detail below with reference to the embodiments and the accompanying drawings.
<实施例><Example>
图1是本发明的实施例中的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习***的结构框图。FIG. 1 is a structural block diagram of a deep learning system for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface characteristics in an embodiment of the present invention.
如图1所示，基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习***100，用于通过拍摄得到的人脸图片中的眼部区域来进行患者的疾病风险筛查，包括人脸图像预处理部1、眼部图像特征提取部2、分类部3、疾病风险筛查评估部4、输出显示部5、***通信部6以及用于控制上述各部的***控制部7。As shown in FIG. 1 , the deep learning system 100 for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface features is used to screen the disease risk of patients by taking the eye area in the obtained face picture , including a face image preprocessing part 1, an eye image feature extraction part 2, a classification part 3, a disease risk screening evaluation part 4, an output display part 5, a system communication part 6 and a system control part 7 for controlling the above parts .
本实施例中，如图2所示，基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习***100由计算机设备110和结果展示设备140组成，用于对外部的媒体数据进行处理。其中，计算机设备由处理器120以及内存130构成：处理器120是一个用于计算以及运行可执行代码的硬件处理器，如中央处理器CPU或图形计算处理器GPU；内存130是一个非易失的存储设备，用于储存可执行代码从而让处理器120执行相应的计算过程。同时，内存130也会存储各类中间数据及参数。内存130存储内容包括存储有人脸图片的人脸数据集及其相关参数、可执行代码，可执行代码用于运行人脸图像预处理部1、眼部图像特征提取部2以及分类部3。In this embodiment, as shown in FIG. 2 , the deep learning system 100 for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface characteristics is composed of a computer device 110 and a result display device 140, and is used for external media data. to be processed. The computer device is composed of a processor 120 and a memory 130: the processor 120 is a hardware processor for computing and running executable codes, such as a central processing unit CPU or a graphics computing processor GPU; the memory 130 is a non-volatile The storage device is used to store executable codes so as to allow the processor 120 to perform corresponding computing processes. Meanwhile, the memory 130 also stores various intermediate data and parameters. The storage content of the memory 130 includes a face data set storing face pictures and related parameters, executable codes, and the executable codes are used to run the face image preprocessing part 1 , the eye image feature extraction part 2 and the classification part 3 .
本实施例中，媒体数据通过各类数据采集设备拍摄得到，如智能手机等，媒体数据可以是视频内容，也可以是图像内容。媒体数据还可以是包含多个不同身份以及患病类型的人脸图像数据，人脸图片从媒体数据中得到。In this embodiment, the media data is captured by various data collection devices, such as a smart phone, and the media data may be video content or image content. The media data may also be face image data including multiple different identities and disease types, and the face pictures are obtained from the media data.
人脸图像预处理部1用于对人脸图片进行预处理并得到眼部区域图片。其中，人脸图像预处理部1具有一个人脸检测模型，人脸图像预处理部1基于人脸检测模型对人脸图片进行预处理，如图3所示，该预处理包括以下步骤:The face image preprocessing part 1 is used to preprocess the face image and obtain the eye region image. Wherein, the face image preprocessing part 1 has a face detection model, and the face image preprocessing part 1 preprocesses the face image based on the face detection model, as shown in Figure 3, the preprocessing comprises the following steps:
步骤S1，通过人脸检测模型获取人脸图片中人脸的位置区域以及面部关键点坐标。In step S1, the position area of the face in the face picture and the coordinates of the key points of the face are obtained through the face detection model.
本实施例中，由于数据采集的灵活性，原始的人脸图片通常不仅包含眼部区域，还包括背景或面部的其他区域，如鼻子、耳朵和嘴巴。如果直接使用原始的人脸图片进行分类，必然会引入噪声或者不相关的信息，比如背景噪声或鼻子的特征，使得眼部图像特征提取模型的性能不准确、不可靠。因此，为了使眼部图像特征提取模型聚焦于提取眼表特征，本实施例采用人脸检测模型获取图片中人脸的位置区域以及面部关键点坐标，该面部关键点坐标中包括人脸左右眼以及面部其他器官的位置坐标。In this embodiment, due to the flexibility of data collection, the original face picture usually includes not only the eye area, but also the background or other areas of the face, such as the nose, ears, and mouth. If the original face image is directly used for classification, it will inevitably introduce noise or irrelevant information, such as background noise or nose features, making the performance of the eye image feature extraction model inaccurate and unreliable. Therefore, in order to make the eye image feature extraction model focus on extracting eye surface features, the present embodiment adopts the face detection model to obtain the position area of the face in the picture and the coordinates of the facial key points, and the facial key point coordinates include the left and right eyes of the human face. And the position coordinates of other organs of the face.
步骤S2，根据面部关键点坐标中的人脸左右眼关键点，得到人脸眼部区域的横纵坐标的最大值和最小值，同时对横纵坐标进行一定数值的向外扩充来确保人脸眼部区域全部纳入特征提取范围内，并对人脸眼部区域进行截取，得到眼部截取图片。Step S2, according to the key points of the left and right eyes of the face in the key point coordinates of the face, obtain the maximum value and the minimum value of the horizontal and vertical coordinates of the eye region of the human face, and at the same time, the horizontal and vertical coordinates are expanded outward by a certain value to ensure the face. The eye area is all included in the feature extraction range, and the eye area of the face is intercepted to obtain an eye interception image.
步骤S3，对眼部截取图片进行筛选，得到眼部区域图片。Step S3: Screen the eye intercepted pictures to obtain eye region pictures.
对眼部截取图片进行筛选时，将眼部截取图片中纵向长度长于横向长度的图片进行剔除后得到眼部区域图片。When screening the eye clipped pictures, the pictures of the eye clipped pictures whose vertical length is longer than the horizontal length are eliminated to obtain the eye region pictures.
本实施例中，考虑到因存在人脸角度、背景噪声等影像，人脸检测模型得到的面部关键点坐标并非完全准确无误，因而人脸眼部区域的截取也随之会有相应的偏差，考虑到人脸眼部区域应为横向较长的长方形区域，因而在横纵长度比上凡纵向长度长于横向的均认为是人脸眼部区域定位失败的结果，因此需将此类眼部截取图片剔除。In this embodiment, considering that there are images such as face angles and background noises, the facial key point coordinates obtained by the face detection model are not completely accurate, so the interception of the face eye region will also have corresponding deviations. Considering that the eye area of the human face should be a rectangular area with a longer horizontal length, any vertical length longer than horizontal in the aspect ratio is considered to be the result of the failure to locate the eye area of the human face, so such eyes need to be intercepted. Image culling.
眼部图像特征提取部2用于通过训练后的眼部图像特征提取模型提取眼部区域图片的基础特征。The eye image feature extraction part 2 is used for extracting the basic features of the eye region picture through the trained eye image feature extraction model.
图4是本发明的实施例中的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习***中训练眼部图像特征提取模型的流程图。4 is a flowchart of training an eye image feature extraction model in the deep learning system for risk screening of patients with novel coronavirus pneumonia (COVID-19) disease based on ocular surface features in an embodiment of the present invention.
如图4所示，眼部图像特征提取模型训练时包括以下步骤：As shown in Figure 4, the training of the eye image feature extraction model includes the following steps:
步骤E1，将眼部区域图片输入眼部图像特征提取模型中提取基础特征，分类部根据基础特征预测眼部区域图片中对应患者的新冠肺炎(COVID-19)患病类别并输出图片级分类结果。Step E1, input the eye area image into the eye image feature extraction model to extract basic features, and the classification unit predicts the new coronary pneumonia (COVID-19) disease category of the corresponding patient in the eye area image according to the basic features and outputs the image-level classification result .
步骤E2，通过构建损失函数，根据图片级分类结果和眼部区域图片对应患者的真实患病类别，对眼部图像特征提取模型提取的基础特征进行监督。Step E2, by constructing a loss function, supervise the basic features extracted by the eye image feature extraction model according to the picture-level classification result and the actual disease category of the patient corresponding to the eye region picture.
步骤E3，采用SGD优化器对眼部图像特征提取模型进行迭代训练。Step E3, using the SGD optimizer to iteratively train the eye image feature extraction model.
分类部3用于根据基础特征进行图片级的新冠肺炎(COVID-19)患病类别预测并得到图片级分类结果，以及用于根据图片级分类结果进行病患级分类得到病患级预测结果。The classification part 3 is used to perform picture-level prediction of the disease category of the novel coronavirus pneumonia (COVID-19) according to the basic features and obtain picture-level classification results, and to perform patient-level classification according to the picture-level classification results to obtain patient-level prediction results.
本实施例中，图片级的患病类别预测为判断眼部区域图片对应的患者是否患有疾病或其他疾病(例如，普通肺炎或者眼部疾病等)。In this embodiment, the picture-level disease category prediction is to determine whether the patient corresponding to the eye region picture has a disease or other disease (for example, common pneumonia or eye disease, etc.).
当分类部3进行病患级分类时，将患病类别中最紧急的疾病设为最高优先级别患病类别，对于一位患者的多张眼部区域图片中，当预测得到的图片级分类结果中的最高优先级别患病类别的个数大于等于1时，判断患者患有该疾病的可能性较大，得到病患级预测结果。When the classification unit 3 performs patient-level classification, the most urgent disease in the disease category is set as the highest-priority disease category. For a patient's multiple eye region pictures, when the predicted picture-level classification result When the number of the highest priority disease category is greater than or equal to 1, it is judged that the patient is more likely to suffer from the disease, and a patient-level prediction result is obtained.
本实施例中，疾病风险筛查评估部4用于根据图片级分类结果和病患级预测结果进行患病风险评估。In this embodiment, the disease risk screening and evaluation unit 4 is configured to perform disease risk evaluation according to the picture-level classification result and the patient-level prediction result.
结果展示设备包括输出显示部5，用于播放媒体数据和显示图片级分类结果以及病患级预测结果，可以是电脑、电视或者移动设备。The result display device includes an output display unit 5 for playing media data and displaying picture-level classification results and patient-level prediction results, which can be a computer, a TV, or a mobile device.
图5是本发明的实施例中的基于眼表特征的疾病患者风险筛查深度学习***的疾病筛查流程示意图，图6是本发明的实施例中的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习***的疾病筛查流程框图。5 is a schematic diagram of a disease screening process flow of the deep learning system for risk screening of disease patients based on ocular surface features in an embodiment of the present invention, and FIG. 6 is an embodiment of the present invention. 19) A block diagram of the disease screening process of the deep learning system for risk screening of disease patients.
如图5和图6所示，本实施例的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习***的疾病筛查流程包括以下步骤：As shown in FIG. 5 and FIG. 6 , the disease screening process of the deep learning system for risk screening of new coronary pneumonia (COVID-19) disease patients based on ocular surface characteristics of this embodiment includes the following steps:
步骤T1，人脸图像预处理部1对拍摄得到的人脸图片进行预处理，得到眼部区域图片，然后进入步骤T2；Step T1, the face image preprocessing part 1 preprocesses the captured face image to obtain an eye region image, and then enters step T2;
步骤T2，眼部图像特征提取部2根据眼部区域图片提取眼部区域图片的基础特征，然后进入步骤T3；Step T2, the eye image feature extraction part 2 extracts the basic features of the eye region picture according to the eye region picture, and then enters step T3;
步骤T3，分类部3根据基础特征进行图片级的患病类别预测，预测图片对应患者是否患有疾病或其他疾病(例如普通肺炎或者眼部疾病等)，得到图片级分类结果，然后进入步骤T4；Step T3, the classification unit 3 performs picture-level disease category prediction according to the basic features, predicts whether the patient has a disease or other disease (such as common pneumonia or eye disease, etc.) corresponding to the picture, obtains the picture-level classification result, and then enters step T4 ;
步骤T4，分类部3将患病类别中最紧急的疾病设为最高优先级别患病类别，当图片级分类结果中的最高优先级别患病类别的个数大于等于1时，判断患者患有该疾病的可能性较大，得到病患级预测结果，然后进入步骤T5；Step T4, the classification unit 3 sets the most urgent disease in the disease category as the highest priority disease class, and when the number of the highest priority disease class in the picture-level classification result is greater than or equal to 1, it is determined that the patient suffers from the disease. The possibility of the disease is high, the patient-level prediction result is obtained, and then the step T5 is entered;
步骤T5，输出显示部5显示图片级分类结果以及病患级预测结果让用户查看，然后进入结束状态。In step T5, the output display unit 5 displays the picture-level classification result and the patient-level prediction result for the user to view, and then enters the end state.
实施例的作用与效果Action and effect of the embodiment
根据本实施例所涉及的基于眼表特征的疾病患者风险筛查深度学习***，因为能够根据疾病患者眼表往往有着类结膜炎的表现，包括结膜充血、化瘀、溢液或分泌物增多等特征，借助深度学习网络的特征提取和分类预测能力，通过眼部图像特征提取模型提取的基础特征来进行图片级以及病患级的患病类别预测，针对人脸眼部区域进行特征的学习表达，通过捕获更具分辨力和识别力的特征，能够实现基于眼部特征的疾病患者风险筛查；并且本发明通过拍摄人脸图片并根据人脸图片中的眼部区域图片就能够进行疾病患者的筛查工作，能够提高疾病风险筛查的快捷性、准确性以及方便性，同时能够摆脱专业 人员依赖等限制，可大规模普及，在疫情阶段能够实现随时随地的定量检测，动态监测病毒感染的程度，观察治疗效果，进行疫情跟踪和疫情地图绘制，从而实现高效的疫情防控。According to the deep learning system for risk screening of disease patients based on ocular surface characteristics involved in this embodiment, because the ocular surface of disease patients often has conjunctivitis-like manifestations, including conjunctival congestion, stasis removal, discharge or increased secretion, etc. Features: With the help of the feature extraction and classification prediction capabilities of the deep learning network, the basic features extracted by the eye image feature extraction model are used to predict the disease category at the picture level and the patient level, and the learning and expression of features for the face and eye area , by capturing more distinguishing and discriminating features, the risk screening of disease patients based on eye features can be realized; and the present invention can carry out disease patients by taking a face picture and according to the eye area picture in the face picture. The screening work can improve the speed, accuracy and convenience of disease risk screening, and at the same time, it can get rid of restrictions such as professional reliance, and can be popularized on a large scale. In the epidemic stage, quantitative detection can be realized anytime and anywhere, and virus infection can be dynamically monitored. To achieve efficient epidemic prevention and control.
进一步地，因为先通过对人脸图片进行预处理并筛选得到眼部区域图片再输入眼部图像特征提取模型，能够确保输入的图片均为定位准确的眼部区域图片，有效去除了人脸图片中的噪声和不相关的信息，保证眼部图像特征提取模型能够正确的处理眼部区域图片。Further, because the image of the eye region is obtained by preprocessing and screening the face image first, and then inputting the eye image feature extraction model, it can be ensured that the input pictures are all the images of the eye region with accurate positioning, and the face image is effectively removed. The noise and irrelevant information in the eye image ensure that the eye image feature extraction model can correctly process the eye region picture.
上述实施方式为本发明的优选案例，并不用来限制本发明的保护范围。The above embodiments are preferred cases of the present invention, and are not intended to limit the protection scope of the present invention.

Claims (4)

  1. 一种基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习***，用于通过拍摄得到的人脸图片中的眼部区域来进行患者的疾病风险筛查，其特征在于，包括：A deep learning system for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface characteristics, which is used to screen the disease risk of patients by taking the eye region in the obtained face picture, characterized in that: include:
    人脸图像预处理部，用于对所述人脸图片进行预处理并得到眼部区域图片；a face image preprocessing part, which is used for preprocessing the face image and obtaining an eye region image;
    眼部图像特征提取部，用于通过训练后的眼部图像特征提取模型提取所述眼部区域图片的基础特征；an eye image feature extraction part, used for extracting the basic features of the eye region picture through the trained eye image feature extraction model;
    分类部，用于根据所述基础特征进行图片级的新冠肺炎(COVID-19)患病类别预测并得到图片级分类结果，以及用于根据所述图片级分类结果进行病患级分类得到新冠肺炎(COVID-19)病患级预测结果，The classification part is used to perform picture-level prediction of the new coronary pneumonia (COVID-19) disease category according to the basic features and obtain picture-level classification results, and to perform patient-level classification according to the picture-level classification results to obtain new coronary pneumonia (COVID-19) patient-level prediction results,
    其中，所述眼部图像特征提取模型训练时包括以下步骤：Wherein, the training of the eye image feature extraction model includes the following steps:
    步骤S1，将所述眼部区域图片输入所述眼部图像特征提取模型中提取所述基础特征，所述分类部根据所述基础特征预测所述眼部区域图片中对应患者的新冠肺炎(COVID-19)患病类别并输出所述图片级分类结果；Step S1, input the eye region picture into the eye image feature extraction model to extract the basic feature, and the classification unit predicts the new coronary pneumonia (COVID-19) of the corresponding patient in the eye region picture according to the basic feature. -19) Disease category and output the picture-level classification result;
    步骤S2，通过构建损失函数，根据所述图片级分类结果和所述眼部区域图片对应患者的真实新冠肺炎(COVID-19)患病类别，对所述眼部图像特征提取模型提取的所述基础特征进行监督；Step S2, by constructing a loss function, according to the picture-level classification result and the real new coronary pneumonia (COVID-19) disease category of the patient corresponding to the eye area picture, the described eye image feature extraction model is extracted. Basic characteristics to supervise;
    步骤S3，采用SGD优化器对所述眼部图像特征提取模型进行迭代训练。Step S3, using the SGD optimizer to iteratively train the eye image feature extraction model.
  2. 根据权利要求1所述的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习***，其特征在于：The deep learning system for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface characteristics according to claim 1, wherein:
    其中，所述人脸图像预处理部中还具有人脸检测模型，所述人脸图像预处理部对所述人脸图片的预处理包括以下步骤：Wherein, the face image preprocessing part also has a face detection model, and the preprocessing of the face image by the face image preprocessing part includes the following steps:
    步骤E1，通过所述人脸检测模型获取所述人脸图片中人脸的位置区域以及面部关键点坐标；Step E1, obtain the position area and facial key point coordinates of the face in the face picture through the face detection model;
    步骤E2，根据所述面部关键点坐标中的人脸左右眼关键点，得到人脸眼部区域的横纵坐标的最大值和最小值，同时对横纵坐标进行一定数值的向外扩充来确保所述人脸眼部区域全部纳入特征提取范围内，并对所述人脸眼部区域进行截取，得到眼部截取图片；Step E2, according to the key points of the left and right eyes of the human face in the coordinates of the facial key points, obtain the maximum value and the minimum value of the horizontal and vertical coordinates of the eye region of the human face, and simultaneously carry out a certain numerical value of the horizontal and vertical coordinates to expand outward to ensure that. All the face and eye regions are included in the feature extraction range, and the face and eye regions are intercepted to obtain an eye intercepted picture;
    步骤E3，对所述眼部截取图片进行筛选，得到所述眼部区域图片。Step E3: Screen the eye intercepted pictures to obtain the eye region pictures.
  3. 根据权利要求2所述的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习***，其特征在于：The deep learning system for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface characteristics according to claim 2, wherein:
    其中，对所述眼部截取图片进行筛选时，将所述眼部截取图片中纵向长度长于横向长度的图片进行剔除后得到所述眼部区域图片。Wherein, when screening the eye clipped pictures, the eye region pictures are obtained after removing pictures whose vertical length is longer than horizontal length in the eye clipped pictures.
  4. 根据权利要求1所述的基于眼表特征的新冠肺炎(COVID-19)疾病患者风险筛查深度学习***，其特征在于：The deep learning system for risk screening of patients with new coronary pneumonia (COVID-19) disease based on ocular surface characteristics according to claim 1, wherein:
    其中，所述分类部进行病患级分类时，将所述患病类别中最紧急的疾病设为最高优先级别患病类别，对于一位患者的多张所述眼部区 域图片中，当预测得到的所述图片级分类结果中的所述最高优先级别患病类别的个数大于等于1时，判断患者患有该疾病的可能性较大，得到所述病患级预测结果。Wherein, when the classification unit performs patient-level classification, the most urgent disease in the disease category is set as the highest priority disease category. For a plurality of pictures of the eye region of a patient, when predicting When the number of the highest priority disease category in the obtained picture-level classification result is greater than or equal to 1, it is determined that the patient is more likely to have the disease, and the patient-level prediction result is obtained.
PCT/CN2020/119241 2020-08-25 2020-09-30 Ocular surface features-based novel coronavirus pneumonia (covid-19) disease patient risk screening deep learning system WO2022041396A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010862461.5A CN114187223A (en) 2020-08-25 2020-08-25 New coronary pneumonia disease patient risk screening deep learning system based on ocular surface characteristics
CN202010862461.5 2020-08-25

Publications (1)

Publication Number Publication Date
WO2022041396A1 true WO2022041396A1 (en) 2022-03-03

Family

ID=80354467

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/119241 WO2022041396A1 (en) 2020-08-25 2020-09-30 Ocular surface features-based novel coronavirus pneumonia (covid-19) disease patient risk screening deep learning system

Country Status (2)

Country Link
CN (1) CN114187223A (en)
WO (1) WO2022041396A1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220164568A1 (en) * 2020-11-20 2022-05-26 Xiamoi Technology (Wuhan) Co., Ltd. Method for behaviour recognition, electronic equipment, and storage medium
CN116705325A (en) * 2023-06-26 2023-09-05 国家康复辅具研究中心 Wound infection risk assessment method and system
US11816924B2 (en) 2020-11-20 2023-11-14 Xiaomi Technology (Wuhan) Co., Ltd. Method for behaviour recognition based on line-of-sight estimation, electronic equipment, and storage medium
CN117877103A (en) * 2024-03-13 2024-04-12 宁波市眼科医院 Intelligent keratitis screening method based on deep meta learning

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106530295A (en) * 2016-11-07 2017-03-22 首都医科大学 Fundus image classification method and device of retinopathy
CN110111316A (en) * 2019-04-26 2019-08-09 广东工业大学 Method and system based on eyes image identification amblyopia

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106530295A (en) * 2016-11-07 2017-03-22 首都医科大学 Fundus image classification method and device of retinopathy
CN110111316A (en) * 2019-04-26 2019-08-09 广东工业大学 Method and system based on eyes image identification amblyopia

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220164568A1 (en) * 2020-11-20 2022-05-26 Xiamoi Technology (Wuhan) Co., Ltd. Method for behaviour recognition, electronic equipment, and storage medium
US11790692B2 (en) * 2020-11-20 2023-10-17 Xiaomi Technology (Wuhan) Co., Ltd. Method for behaviour recognition, electronic equipment, and storage medium
US11816924B2 (en) 2020-11-20 2023-11-14 Xiaomi Technology (Wuhan) Co., Ltd. Method for behaviour recognition based on line-of-sight estimation, electronic equipment, and storage medium
CN116705325A (en) * 2023-06-26 2023-09-05 国家康复辅具研究中心 Wound infection risk assessment method and system
CN116705325B (en) * 2023-06-26 2024-01-19 国家康复辅具研究中心 Wound infection risk assessment method and system
CN117877103A (en) * 2024-03-13 2024-04-12 宁波市眼科医院 Intelligent keratitis screening method based on deep meta learning
CN117877103B (en) * 2024-03-13 2024-05-24 宁波市眼科医院 Intelligent keratitis screening method based on deep meta learning

Also Published As

Publication number Publication date
CN114187223A (en) 2022-03-15

Similar Documents

Publication Publication Date Title
WO2022041396A1 (en) Ocular surface features-based novel coronavirus pneumonia (covid-19) disease patient risk screening deep learning system
WO2020151489A1 (en) Living body detection method based on facial recognition, and electronic device and storage medium
WO2020215552A1 (en) Multi-target tracking method, apparatus, computer device, and storage medium
WO2019033574A1 (en) Electronic device, dynamic video face recognition method and system, and storage medium
WO2020042541A1 (en) Eyeball tracking interactive method and device
WO2020042542A1 (en) Method and apparatus for acquiring eye movement control calibration data
CN111598038B (en) Facial feature point detection method, device, equipment and storage medium
CN112287867B (en) Multi-camera human body action recognition method and device
JP2013504114A (en) Eye state detection apparatus and method
CN112287868B (en) Human body action recognition method and device
WO2021139167A1 (en) Method and apparatus for facial recognition, electronic device, and computer readable storage medium
WO2022213349A1 (en) Method and apparatus for recognizing face with mask, and computer storage medium
CN115496700A (en) Disease detection system and method based on eye image
WO2021208767A1 (en) Facial contour correction method and apparatus, and device and storage medium
US11132778B2 (en) Image analysis apparatus, image analysis method, and recording medium
CN106778751B (en) Non-facial ROI (region of interest) identification method and device
WO2023231479A1 (en) Pupil detection method and apparatus, and storage medium and electronic device
WO2022142368A1 (en) Rapid screen system based on eye region image
CN113435361B (en) Mask identification method based on depth camera
CN116484916A (en) Chicken health state detection method and method for building detection model thereof
JP2003178304A (en) Face image retrieving device, face image retrieving method and program for executing method on computer
WO2021078037A1 (en) Facial skin detection method and apparatus
WO2022213348A1 (en) Recognition method and apparatus for detecting face with mask, and computer storage medium
WO2021259033A1 (en) Facial recognition method, electronic device, and storage medium
CN115311723A (en) Living body detection method, living body detection device and computer-readable storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20951064

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 090823)

122 Ep: pct application non-entry in european phase

Ref document number: 20951064

Country of ref document: EP

Kind code of ref document: A1