WO2019206209A1 - 基于机器学习的眼底图像检测方法、装置及*** - Google Patents

基于机器学习的眼底图像检测方法、装置及*** Download PDF

Info

Publication number
WO2019206209A1
WO2019206209A1 PCT/CN2019/084210 CN2019084210W WO2019206209A1 WO 2019206209 A1 WO2019206209 A1 WO 2019206209A1 CN 2019084210 W CN2019084210 W CN 2019084210W WO 2019206209 A1 WO2019206209 A1 WO 2019206209A1
Authority
WO
WIPO (PCT)
Prior art keywords
feature
fundus image
classification model
classification
region
Prior art date
Application number
PCT/CN2019/084210
Other languages
English (en)
French (fr)
Inventor
熊健皓
赵昕
马永培
李舒磊
和超
张大磊
Original Assignee
上海鹰瞳医疗科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海鹰瞳医疗科技有限公司 filed Critical 上海鹰瞳医疗科技有限公司
Priority to EP19792203.2A priority Critical patent/EP3785603B1/en
Publication of WO2019206209A1 publication Critical patent/WO2019206209A1/zh

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/14—Arrangements specially adapted for eye photography
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/12—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for looking at the eye fundus, e.g. ophthalmoscopes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G06V10/464—Salient features, e.g. scale invariant feature transforms [SIFT] using a plurality of salient features, e.g. bag-of-words [BoW] representations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/18—Extraction of features or characteristics of the image
    • G06V30/18143—Extracting features based on salient regional features, e.g. scale invariant feature transform [SIFT] keypoints
    • G06V30/18152—Extracting features based on a plurality of salient regional features, e.g. "bag of words"
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/193—Preprocessing; Feature extraction

Definitions

  • the present invention relates to the field of medical image recognition technology, and in particular to a method, device and system for detecting fundus images based on machine learning.
  • deep learning technology can detect a certain feature of the fundus image more accurately.
  • a deep learning model is trained by using a large number of samples of the macular hole, and the fundus image is used to perform a yellow spot on the fundus image.
  • Split hole detection These techniques are often limited to the detection of a single feature or a small number of associated features, and cannot be accurately detected for other features.
  • the eye is a very fine and complex organ in the human body, it contains a wide variety of features, and the differences between the features are often large.
  • each of the features is trained to perform a separate test, which not only requires a large number of samples, but also causes a large increase in the number of features, resulting in a decrease in the detection efficiency.
  • a method for detecting a fundus image based on machine learning comprising:
  • the detection result is determined based on at least the classification result of the first classification model and the second classification model.
  • a machine vision-based fundus image detecting apparatus including:
  • a first classification model configured to classify an entire area of the fundus image, and determine whether the first feature is included in the fundus image
  • At least one second classification model configured to classify a specific region in the fundus image, and determine whether the second feature is included in the fundus image, wherein a degree of significance of the first feature is greater than a degree of significance of the second feature;
  • a decision module configured to determine the detection result according to at least the classification result of the first classification model and the second classification model.
  • an electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the one processor, The instructions are executed by the at least one processor to cause the at least one processor to perform the machine learning based fundus image detection method of the first aspect.
  • a computer storage medium having instructions stored thereon that, when executed on a computer, cause the computer to perform the fundus image detection method of the first aspect.
  • a computer program product comprising instructions which, when run on a computer, cause the computer to perform the fundus image detection method of the first aspect.
  • a fundus image detection system based on machine learning including:
  • Image acquisition device for collecting fundus images
  • the electronic device of the second aspect configured to communicate with the image capturing device, for detecting the fundus image
  • the output device is in communication with the fundus image detecting device for outputting a detection result of the fundus image.
  • the entire region of the fundus image to be detected is detected with the first feature having a high degree of significance, and the specific region of the fundus image is synchronized significantly.
  • the low second feature detection enables the two classification models to not interfere with each other and has a clear division of labor, and can accurately classify each region to determine whether or not the relevant features are included, and finally combine the classification results of the two features to determine the detection result. Improve the accuracy of the final test results, thereby achieving simultaneous detection of multiple categories and multiple saliency features, with high efficiency.
  • FIG. 1 is a flow chart showing a method for detecting a fundus image based on machine learning
  • FIG. 2 is a flow chart showing another method for detecting fundus image based on machine learning
  • FIG. 3 is a schematic diagram of a fundus image detecting apparatus based on machine learning
  • Figure 4 shows a schematic view of a preferred fundus image detecting device
  • FIG. 5 shows a schematic diagram of an electronic device for performing a fundus image detecting method
  • Fig. 6 shows a schematic diagram of a fundus image detection system based on machine learning.
  • Embodiments of the present invention provide a fundus image detection method based on machine learning, which can be executed by a computer or a server. As shown in FIG. 1, the method includes the following steps:
  • This image is a fundus retina image taken by an eye detecting device for diagnosing an eye lesion.
  • the first classification model uses the first classification model to classify the entire area of the fundus image, and determining whether the first feature is included in the fundus image.
  • the first feature is a feature with a high degree of significance.
  • the so-called saliency can be weighed by factors such as color chromatic aberration, contrast and gradation, or the size of the occupied area. For example, in an entire area of the fundus image, there is a region having a large difference from the normal fundus color, and the ratio of the region is larger than a certain preset value, and the region is taken as the first feature.
  • the first feature may include a large area of abnormal tissue or structure within the fundus, a large spot in the fundus range, and the like, such as image features associated with leopard-like fundus, fundus leukoplakia, fundus laser spot, and the like.
  • the scheme uses a machine learning algorithm to perform the first feature detection. Before applying to the detection, a plurality of fundus image samples having various first features should be used to train the classification model to have a certain classification ability.
  • the first classification model may be a single classification model or a multi-class classification model. If it is a single classification model, the output result is two types, that is, with or without the first feature; if it is a multi-class model, the output result is multi-class, that is, does not include The first feature or the category of the first feature included.
  • Steps S12 and S13 are preferably performed synchronously or in any order.
  • the second feature should be understood as a detail feature, the saliency of the first feature being greater than the saliency of the second feature.
  • the color chromatic aberration, contrast, gradation, or area of the second feature is smaller than the color chromatic aberration, contrast, gradation, or area of the first feature.
  • the first classification model and the second classification model both detect the second feature, but the sensitivity of the first classification model to the second feature is relatively low, and The sensitivity of the two-category model will be higher.
  • the second feature is in a specific region including a disc region, a macula region, and at least one of a blood vessel region and a retina region, and may be one region or a plurality of regions of the set range.
  • the specific area is the optic disc area
  • the second feature includes a specific feature such as an abnormal shape of the optic disc, an abnormal color of the optic disc, an abnormality of the optic nerve, or the specific area is a macular area
  • the second characteristic includes a specific feature such as a macular structural abnormality and a macular shape abnormality
  • the specific region is a blood vessel region
  • the second feature includes a specific feature such as abnormal blood color, abnormal blood vessel orientation, abnormal shape of the central vein, abnormal shape of the branch vein, or the specific region is a retinal region
  • the second feature includes a small abnormal point such as a color. Anomalous points, irregularly shaped points, or reduced retinal areas.
  • the second feature may also include features of other details in the fundus, such as vascular lines and the
  • the scheme uses a machine learning algorithm to perform the second feature detection. Before applying to the detection, a plurality of fundus image samples having various second features should be used to train the corresponding classification model to have a certain classification ability.
  • a plurality of second classification models are used to perform parallel classification detection for different specific regions, and each of the second classification models independently outputs classification results.
  • the classification model A detects the specific features associated with the optic disc, such as papillary edema, optic discitis, optic atrophy, and the like, for the optic disc region;
  • Model B is for the macular area to detect whether it contains specific features related to the macula, such as macular hole, macular edema, macular area atrophy and other characteristics of various macular lesions;
  • classification model C for the blood vessel area, to detect whether it contains blood vessels Related specific features, such as vitreous hemorrhage, choroidal hemangioma, central venous obstruction, branch vein occlusion and other characteristics of various macular degeneration.
  • the second classification model can be set to output a bi-classification result to indicate the presence or absence of a second feature of the fundus image.
  • the second classification model may be set to output a multi-category result to indicate that the fundus image does not have a second feature, or a specific category of the included second feature.
  • whether the output of the multi-category result or the single-category result may be set according to whether there is a conflict between the specific categories of the second classification model.
  • S14 Determine the detection result according to at least the classification result of the first classification model and the second classification model.
  • the classification results of the two classification models can be completely followed, that is, the classification results of the two classification models can be directly output, and the classification results of the two classification models can also be judged to obtain the final detection result.
  • the so-called judgment refers to determining a comprehensive detection result based on the combination of the classification results output by the first classification model and the second classification model, and the final detection result may be inconsistent with the classification result output by the two classification models.
  • the classification results of the first classification model and the second classification model are classified by a machine learning algorithm, and the classification result is used as a final detection result.
  • a decision model is introduced, which is also a classification model, specifically a two-class model or a multi-class model.
  • the input data of the decision model is a kind of tag information rather than a fundus image, and the content of the tag information is whether there is a first feature and a second feature, or a specific category of the first feature and the second feature.
  • the classification result output by the classification model is generally a numerical value, specifically a confidence information or probability expressed by 0-1.
  • the program can use the output value as the final test result, and can further judge the value, and determine the corresponding test result according to the size of the value.
  • the decision model Before applying to the decision, the decision model should be trained with a large amount of sample data to have a certain classification ability. Regarding the sample data, information including whether the first feature and the second feature and their corresponding tags, or information including the first feature and the second feature specific category, and their corresponding tags should be included.
  • the fundus image may be subjected to any one or any combination of stain/bright spot detection, exposure detection, sharpness detection, light leakage detection, and local shadow detection.
  • the stain/bright spot detection adopts weighted averaging processing on the plurality of to-be-detected images to obtain an average image, thereby determining whether there is a pixel point exceeding the preset brightness range in the average image; when there is more than a preset brightness range in the average image At the pixel point, it is confirmed that there is a stain/bright spot in the image to be detected.
  • the detection of stains or bright spots can be done.
  • the image to be detected is binarized to obtain a preset region in the image; a mask based on the boundary of the preset region is generated; the mask is used to merge with the image to be detected; and the average color brightness of the image after fusion is obtained. And compared with the preset color brightness threshold; according to the comparison result, the degree of light leakage of the image to be detected is confirmed. When the degree of light leakage is greater than the preset value, it can be confirmed that the fundus image is leaking light.
  • the histogram of any color channel in the image to be detected is counted; the number of pixels smaller than the preset pixel value is counted; whether the number of pixels smaller than the preset pixel value is less than the preset number; When the number of pixel points of the pixel value is less than the preset number, it is confirmed that there is a local shadow in the image to be detected.
  • the high frequency component of the image to be detected is extracted; the information amount of the high frequency component is calculated; and the sharpness of the image to be detected is confirmed based on the information amount of the high frequency component.
  • the image to be detected is converted into a grayscale image; the root mean square of the histogram of the grayscale image is counted; and the exposure of the image to be detected is confirmed based on the root mean square size.
  • the detection result of the image may be affected, resulting in inaccurate detection results. Therefore, in order to ensure the detection accuracy of the image, the image having the above-mentioned quality defect can be eliminated before the classification operation is performed.
  • the embodiment of the present invention further provides a method for detecting fundus images, as shown in FIG. 2, the method includes the following steps:
  • Step S24 Using the third classification model to classify the entire region in the fundus image, determining whether the third feature is included in the fundus image, and the third feature is less prominent than the second feature.
  • the third feature is a more elaborate feature than the second feature, such as a distributed lesion feature such as a minor lesion of the fundus. Steps S22, S23, and S24 are preferably performed synchronously or in any order.
  • the third classification model can be set to output a bi-classification result to indicate the presence or absence of a third feature of the fundus image.
  • the third classification model may be set to output a multi-category result to indicate that the fundus image does not have a third feature, or a specific category of the included third feature.
  • the input data of the decision model is whether there is a first feature, a second feature, and a third feature, or a specific category of the first feature, the second feature, and the third feature.
  • a convolutional neural network is specifically implemented.
  • the basic unit of the convolutional neural network employed is the convolutional layer superposition activation function (ReLu) layer and pooling.
  • the convolution layer is to screen specific image features
  • the activation function layer uses the ReLu activation function to nonlinearly process the selected features
  • the pooling layer uses maximum pooling to maximize the information at different locations. Extract it out.
  • the normalization of the network can be used to improve the network capacity and prevent the gradient dispersion during the training network.
  • the number of network layers per module varies from 15 to 100 layers depending on the type of fundus features that need to be detected.
  • the convolutional neural network implementation may be as follows: input layer -C1-BN1-R1-P1-C2-BN2-R2-P2-C3-BN3-R3-P3-C4-BN4-R4-P4-C5- BN5–R5–P5-FC1–FC2–SoftMax.
  • the input layer is an image of a certain size
  • C represents a convolution layer (same as C1, C2, C3, C4, C5)
  • BN represents a batch normalization layer (same as BN1, BN2, BN3, BN4, BN5)
  • R represents the function activation layer (same reason R1, R2, R3, R4, R5)
  • P represents the pooling layer (same reason P1, P2, P3, P4, P5)
  • the full connection layer is FC1 and FC2, and SoftMax provides output.
  • the convolutional neural network used in the present embodiment is not limited to the structure of the convolutional neural network described above, and other neural network results satisfying the present embodiment are equally applicable.
  • the hidden layer size of the neural network can be changed according to the saliency of the feature, which is the layer between the input and the output. Specifically, features with a large degree of significance use a smaller hidden layer, and features with a small degree of significance use a larger hidden layer. The largest hidden layer of the convolutional network for the second feature and the third feature set that are less significant is larger than the largest hidden layer of the convolutional network of the first feature.
  • the maximum hidden layer size of the network is required to be small, for example, less than 200 ⁇ 200, in order to extract features.
  • the output of the largest size hidden layer should be kept large, such as greater than 300x300, ensuring that fine eye sub-features such as small exudation points and bleeding points can be extracted.
  • the size of the output of the hidden layer of the largest size is determined by the image input layer, the volume base layer, and the pooling layer, and is implemented in various ways, and will not be described herein.
  • the embodiment of the present invention provides a fundus image detecting device based on machine learning.
  • the detecting device includes: an acquiring module 10, configured to acquire a fundus image to be detected; and a first classification model 20 for The entire area of the fundus image is classified to determine whether the first feature is included in the fundus image; at least one second classification model 30 is configured to classify a specific region in the fundus image to determine whether the fundus image includes the second feature, wherein The saliency of a feature is greater than the saliency of the second feature; the decision module 40 is configured to determine the detection result according to at least the classification result of the first classification model and the second classification model.
  • the specific area and the second classification model 30 are respectively multiple, and the plurality of second classification models are respectively used to classify different specific areas and output a second classification result, and the second classification result is used for Indicates whether the fundus image contains a second feature associated with a particular region.
  • the first classification model and the second classification model are both multi-category models, and the classification result is used to indicate whether the first feature and the second feature are included in the fundus image, and the first feature and the second feature. Specific category.
  • the detecting device further includes:
  • a third classification model 50 configured to classify an overall region in the fundus image, and determine whether a third feature is included in the fundus image, and the third feature has a significance less than a second feature;
  • the decision module 40 determines the detection result based on the classification results of the first classification model 20, the second classification model 30, and the third classification model 50.
  • the third classification model is a multi-classification model, and the classification result is used to indicate whether the third feature is included in the fundus image, and the specific category of the third feature.
  • the particular region comprises at least one of a optic disc region, a macula region, a vascular region, and a retinal region.
  • the first feature, the second feature, and the third feature are both fundus lesion features.
  • An electronic device which may be a server or a terminal. As shown in FIG. 5, the controller is included, and the controller includes one or more processors 41 and a memory 42, and one processor 43 is taken as an example in FIG.
  • the electronic device may also include an input device 43 and an output device 44.
  • the processor 41, the memory 42, the input device 43, and the output device 44 may be connected by a bus or other means, as exemplified by a bus connection in FIG.
  • the processor 41 can be a Central Processing Unit (CPU).
  • the processor 41 can also be another general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or Other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., or a combination of the above various types of chips.
  • the general purpose processor may be a microprocessor or the processor or any conventional processor or the like.
  • the memory 42 is a non-transitory computer readable storage medium for storing non-transitory software programs, non-transitory computer executable programs, and modules.
  • the processor 41 executes the various functional applications and data processing of the server by running the non-transitory software programs, instructions, and modules stored in the memory 42, that is, the fundus image detecting method in the above method embodiments.
  • the memory 42 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application required for at least one function; the storage data area may store data created according to use of the processing device operated by the server, and the like.
  • memory 42 can include high speed random access memory, and can also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device.
  • memory 42 may optionally include memory remotely located relative to processor 41, which may be connected to the network connection device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
  • Input device 43 can receive input numeric or character information and generate key signal inputs related to user settings and function control of the processing device of the server.
  • Output device 44 may include a display device such as a display screen.
  • One or more modules are stored in the memory 42, and when executed by one or more processors 41, the method as shown in FIG. 1 or 2 is performed.
  • An embodiment of the present invention further provides a fundus image detection system based on machine learning.
  • the system includes: an image acquisition device 100 for collecting a fundus image.
  • the image collection device may be plural.
  • the image collection device 100 is a fundus camera device of each hospital, or a fundus camera device of an individual user.
  • the fundus detection system further includes a cloud server 200.
  • the cloud server 200 is provided with a fundus image detecting device for performing the above-mentioned fundus image detecting method, and communicates with the graphic capturing device 100, for example, in the form of wireless communication, or adopts In the form of wired communication, the fundus image collected by the image capturing device 100 is uploaded into the cloud server 200, and the detection result is obtained by performing the fundus image detecting method by the electronic device, and the detection result is output through the output device.
  • the output device 300 may be a display device. It can also be printed for the printing device in the form of a report, or it can be a user's terminal device, such as a mobile phone, tablet or personal computer.
  • embodiments of the present invention can be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or a combination of software and hardware. Moreover, the invention can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.
  • computer-usable storage media including but not limited to disk storage, CD-ROM, optical storage, etc.
  • the computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device.
  • the apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.
  • These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device.
  • the instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • General Health & Medical Sciences (AREA)
  • Ophthalmology & Optometry (AREA)
  • Veterinary Medicine (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Surgery (AREA)
  • Public Health (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Animal Behavior & Ethology (AREA)
  • Medical Informatics (AREA)
  • Data Mining & Analysis (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Human Computer Interaction (AREA)
  • Eye Examination Apparatus (AREA)
  • Image Analysis (AREA)

Abstract

一种基于机器学习的眼底图像检测方法、装置及***，方法包括如下步骤：获取待检测的眼底图像（S11）；利用第一分类模型（20）针对眼底图像的整体区域进行分类，确定眼底图像中是否包含第一特征（S12）；以及利用至少一个第二分类模型（30）针对眼底图像中的特定区域进行分类，确定眼底图像中是否包含第二特征，其中第一特征的显著度大于第二特征的显著度（S13）；至少根据第一分类模型（20）和第二分类模型（30）的分类结果确定检测结果（S14）。

Description

基于机器学习的眼底图像检测方法、装置及***
本申请要求于2018年04月26日提交中国专利局、申请号为201810387484.8、申请名称为“基于机器学习的眼底图像检测方法、装置及***”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及医疗图像识别技术领域，具体涉及到一种基于机器学习的眼底图像检测方法、装置及***。
背景技术
近年来机器学习在医学领域得到了广泛的应用，尤其以深度学习为代表的机器学习技术在医疗影像领域被广泛关注。例如，在眼底图像检测方面，深度学习技术可以较为准确的检测出眼底图像某一特征，如利用大量黄斑裂孔这一特征的样本对深度学习模型进行训练，利用训练后的模型对眼底图像进行黄斑裂孔检测。这些技术往往只局限于单一特征或者少量关联特征的检测，对其他特征不能准确的检测。然而，由于眼睛是人体中一个非常精细且复杂的器官，其包含的特征种类繁多，并且各个特征之间的差别往往也比较大。因此，采用现有的检测技术往往会导致检测结果难以收敛，致使检测结果不够准确。或者，将每一个特征均训练一个模型分别进行检测，不仅所需的样本数量巨大，在特征数量较大的情况下，导致计算量急剧提升，进而导致检测效率下降。
因此如何快速准确的对眼底图像进行检测成为亟待解决的技术问题。
发明内容
第一方面，提供了一种基于机器学习的眼底图像检测方法，包括：
获取待检测的眼底图像；
利用第一分类模型针对所述眼底图像的整体区域进行分类，确定所述眼底图像中是否包含第一特征；以及利用至少一个第二分类模型针对所述眼底图像中的特定区域进行分类，确定所述眼底图像中是否包含第二特征，其中所述第一特征的显著度大于第二特征的显著度；
至少根据所述第一分类模型和所述第二分类模型的分类结果确定检测结果。
第二方面，提供了一种基于机器学习的眼底图像检测装置，包括：
获取模块，获取待检测的眼底图像；
第一分类模型，用于针对所述眼底图像的整体区域进行分类，确定所述眼底图像中是否包含第一特征；
至少一个第二分类模型，用于针对所述眼底图像中的特定区域进行分类，确定所述眼底图像中是否包含第二特征，其中所述第一特征的显著度大于第二特征的显著度；
决策模块，用于至少根据所述第一分类模型和所述第二分类模型的分类结果确定检测结果。
第三方面，提供了一种电子设备，包括至少一个处理器；以及与所述 至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器执行第一方面所述的基于机器学习的眼底图像检测方法。
第四方面，提供了一种计算机存储介质，其上存储有指令，当所述指令在计算机上运行时，使得所述计算机执行第一方面所述的眼底图像检测方法。
第五方面，提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行第一方面所述的眼底图像检测方法。
第六方面，提供了一种基于机器学习的眼底图像检测***，包括：
图像采集装置，用于采集眼底图像；以及
第二方面所述的电子设备，与所述图像采集装置通信，用于对所述眼底图像进行检测；
输出装置，与所述眼底图像检测装置通信，用于输出眼底图像的检测结果。
根据本申请提供的基于机器学习的眼底图像检测方法、装置及***，对待检测的眼底图像的整体区域进行显著度较高的第一特征的检测，并同步对眼底图像的特定区域进行显著度较低的第二特征检测，使两种分类模型互不干扰且分工明确，能够精确地对各个区域进行分类而确定是否包含相关的特征，最后结合两种特征的分类结果进行判决得到检测结果，以提高最终检测结果的准确性，由此实现同时进行多种类别、多种显著度的特 征检测，具有较高的效率。
附图说明
为了更清楚地说明本发明具体实施方式或现有技术中的技术方案，下面将对具体实施方式或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图是本发明的一些实施方式，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1示出了一种基于机器学习的眼底图像检测方法的流程图；
图2示出了另一种基于机器学习的眼底图像检测方法的流程图；
图3示出了一种基于机器学习的眼底图像检测装置的示意图；
图4示出了一种优选的眼底图像检测装置的示意图；
图5示出了一种用于执行眼底图像检测方法的电子设备示意图；
图6示出了一种基于机器学习的眼底图像检测***的示意图。
具体实施方式
下面将结合附图对本发明的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
在本发明的描述中，需要说明的是，术语“第一”、“第二”、“第三”仅用于描述目的，而不能理解为指示或暗示相对重要性。此外，下面所描述的本发明不同实施方式中所涉及的技术特征只要彼此之间未构成冲突就可以相互结合。
本发明实施例提供了一种基于机器学习的眼底图像检测方法，可以被计算机或服务器执行。如图1所示，该方法包括如下步骤：
S11.获取待检测的眼底图像。该图像是通过眼部检测设备拍摄的用于诊断眼部病变的眼底视网膜图像。
S12.利用第一分类模型针对眼底图像的整体区域进行分类，确定眼底图像中是否包含第一特征。第一特征是显著度较高的图像特征，所称显著度可以通过颜色色差，对比度和灰度或者所占区域的大小等因素进行权衡。例如在眼底图像整体区域中存在与正常眼底颜色差别较大的区域，并且该区域所占的比例大于某一预设值，则将该区域作为第一特征。
具体地，第一特征可包括眼底范围内较大面积的异常组织或结构、眼底范围内较大的斑点等内容，例如与豹纹状眼底、眼底白斑、眼底激光斑等病变相关的图像特征。
本方案使用机器学习算法进行第一特征检测，在应用于检测前，应使用大量具有各种第一特征的眼底图像样本对分类模型进行训练，使其具有一定的分类能力。第一分类模型可以是单分类模型或者多分类模型，如果是单分类模型则输出结果为两类，即包含或者不包含第一特征；如果是多 分类模型则输出结果是多类，即不包含第一特征或者所包含第一特征的类别。
S13.利用至少一个第二分类模型针对眼底图像中的特定区域进行分类，确定眼底图像中是否包含第二特征，其中第一特征的显著度大于第二特征的显著度。步骤S12和步骤S13优选为同步执行，或者以任意顺序执行。
第二特征应被理解为细节特征，第一特征的显著度大于第二特征的显著度。例如第二特征的颜色色差、对比度、灰度或区域面积小于第一特征的颜色色差、对比度、灰度或区域面积。实际上，因为第一分类模型是针对图像全局进行分类，所以第一分类模型和第二分类模型都会检测到第二特征，但是第一分类模型对第二特征的敏感度会比较低，而第二分类模型的敏感度会比较高。
第二特征处于特定区域中，特定区域包括视盘区域、黄斑区域以及血管区域和视网膜区域中的至少之一个区域，也可以是设定范围的一个区域或多个区域。例如特定区域为视盘区域，则第二特征包括视盘形状异常、视盘颜色异常、视神经异常等具体特征；或者，特定区域为黄斑区域，第二特征包括黄斑结构异常、黄斑形状异常等具体特征；或者，特定区域为血管区域，第二特征包括血管颜色异常、血管走向异常、中央静脉形状异常、分支静脉形状异常等具体特征；或者，特定区域为视网膜区域，第二特征包括细小异常点，例如颜色异常的点、不规则形状的点，或者视网膜区域减小等特征。此外，第二特征也可以包括眼底中其他细节的特征，例如血管纹路等。
本方案使用机器学习算法进行第二特征检测，在应用于检测前，应使用大量具有各种第二特征的眼底图像样本对相应的分类模型进行训练，使其具有一定的分类能力。
在一个实施例中，使用多个第二分类模型分别针对不同的特定区域进行并行分类检测，各个第二分类模型独立输出分类结果。具体地，例如使用三个第二分类模型：分类模型A针对视盘区域，检测其中是否包含与视盘相关的具体特征，如视***水肿、视盘炎、视神经萎缩等各种视盘类病变的特征；分类模型B针对黄斑区域，检测其中是否包含与黄斑相关的具体特征，如黄斑裂孔、黄斑水肿、黄斑区地图萎缩等各种黄斑类病变的特征；分类模型C针对血管区域，检测其中是否包含与血管相关的具体特征，如玻璃体积血、脉络膜血管瘤、中央静脉阻塞、分支静脉阻塞等各种黄斑类病变的特征。
为减小计算量，可以将第二分类模型设置为输出二分类结果，以表示眼底图像存在或者不存在第二特征。为提高检测的精度，可以将第二分类模型设置为输出多分类结果，以表示眼底图像不存在第二特征，或者所包含的第二特征的具体类别。实际应用时，可根据第二分类模型所针对各种具体类别是否存在冲突，来设定其输出多分类结果还是单分类结果。
S14.至少根据第一分类模型和第二分类模型的分类结果确定检测结果。在此步骤中可以完全遵循这两种分类模型的分类结果，即直接输出它们的分类结果，也可以对这两种分类模型的分类结果进行判决得到最终检测结果。所谓判决是指根据第一分类模型和第二分类模型所输出分类结果的组 合，确定一种综合的检测结果，最终的检测结果与两种分类模型输出的分类结果可能是不一致的。
在一个实施例中，通过机器学习算法对第一分类模型和第二分类模型的分类结果进行分类，将分类结果作为最终的检测结果。在此步骤引入了一个判决模型，该模型也是一个分类模型，具体为二分类模型或者多分类模型。需要说明的是，判决模型的输入数据是一种标签信息而非眼底图像，标签信息的内容为是否具有第一特征和第二特征，或者第一特征和第二特征的具体类别。
分类模型输出的分类结果一般是数值，具体是以0-1表达的置信度信息或概率。本方案可以将其输出的数值作为最终的检测结果，也可以对数值做进一步判断，根据数值的大小确定相应的检测结果。
在应用于判决之前，应当使用大量样本数据对判决模型进行训练，以使其具备一定的分类能力。关于样本数据，其中应当包括是否具有第一特征和第二特征的信息及其对应的标签，或者是包括第一特征和第二特征具体类别的信息及其对应的标签。
对待检测的眼底图像的整体区域进行显著度较高的第一特征的检测，并同步对眼底图像的特定区域进行显著度较低的第二特征检测，使两种分类模型互不干扰且分工明确，能够精确地对各个区域进行分类而确定是否包含相关的特征，最后结合两种特征的分类结果进行判决得到检测结果，以提高最终检测结果的准确性，由此实现同时进行多种类别、多种显著度 的特征检测，具有较高的效率。
由于图像拍摄者拍摄的眼底照片的图片质量差异很大，图片常常会曝光过度、灰暗、模糊，这使得机器学习判断的难度大大增加。作为可选地实施例，通过对图像质量进行检测，筛选出质量合格的图像进一步保证图像检测的精度。在具体的实施例中，可以对眼底图像进行污点/亮斑检测、曝光度检测、清晰度检测、漏光检测、局部阴影检测中的任意一种或任意组合。
具体地，污点/亮斑检测采用对多个待检测图像进行加权平均处理，得到平均图像，进而判断平均图像中是否存在超过预设亮度范围的像素点；当平均图像中存在超过预设亮度范围的像素点时，确认待检测图像中存在污点/亮斑。即可完成对污点或亮斑的检测。
关于漏光检测，对待检测图像进行二值化处理，得到图像中的预设区域；生成基于预设区域边界的掩膜；使用掩膜与待检测图像融合；求取融合后图像的平均色彩亮度，并与预设色彩亮度阈值进行对比；根据对比结果确认待检测图像的漏光程度。在漏光程度大于预设值时，可以确认该眼底图像漏光。
关于局部阴影检测，统计待检测图像中任意一个颜色通道的直方图；统计小于预设像素值的像素点的数量；判断小于预设像素值的像素点的数量是否小于预设数量；当小于预设像素值的像素点的数量小于预设数量时，确认待检测图像中存在局部阴影。
关于清晰度检测，提取待检测图像的高频分量；计算高频分量的信息量；基于高频分量的信息量确认待检测图像的清晰度。
关于曝光度检测，将待检测图像转为灰度图像；统计灰度图像直方图的均方根；基于均方根大小确认待检测图像的曝光度。
当眼底图像存在上述质量问题时，可能会影响图像的检测结果，导致检测结果不够准确。因此，为保证图像的检测精度，可以在执行分类操作前，剔除存在上述质量缺陷的图像。
在实际应用中，眼底图像中的一些特征，特别是一些显著度较小的异常特征可能不存在于眼底图像的特定区域，只对特定区域中的显著度较小的特征进行检测，可能会造成漏检，为提高检测的全面性和精度，本发明实施例还提供了一种眼底图像检测方法，如图2所示，该方法包括如下步骤：
S21.获取待检测的眼底图像。
S22.利用第一分类模型针对眼底图像的整体区域进行分类，确定眼底图像中是否包含第一特征。具体参见上述实施例中步骤S12中关于第一特征检测的描述。
S23.利用至少一个第二分类模型针对眼底图像中的特定区域进行分类，确定眼底图像中是否包含第二特征。具体参见上述实施例步骤S13中关于第二特征检测的描述。
S24.利用第三分类模型针对眼底图像中的整体区域进行分类，确定眼底图像中是否包含第三特征，第三特征的显著度小于第二特征的显著度。第三特征是相比于第二特征更为精细的特征，例如眼底的细小损伤等分布式的病变特征。步骤S22、步骤S23和步骤S24优选为同步执行，或者以任意顺序执行。
为减小计算量，可以将第三分类模型设置为输出二分类结果，以表示眼底图像存在或者不存在第三特征。为提高检测的精度，可以将第三分类模型设置为输出多分类结果，以表示眼底图像不存在第三特征，或者所包含的第三特征的具体类别。
S25.根据第一分类模型、第二分类模型和第三分类模型的分类结果确定检测结果。参照上述实施例步骤S14中关于检测结果的描述，在此进一步增加了第三特征的分类结果，使得检测结果更加准确。
在使用判决模型确定最终检测结果时，判决模型的输入数据为是否具有第一特征、第二特征和第三特征，或者第一特征、第二特征和第三特征的具体类别。
关于上述各种分类模型，具体采用卷积神经网络来实现。所采用的卷积神经网络的基本单元机构是卷积层叠加激活函数(ReLu)层和池化(Pooling)。其中，卷积层是对特定的图像特征进行筛选，激活函数层使用ReLu激活函数对筛选出的特征进行非线性处理，池化层使用最大池化(max pooling)将不同位置的最强的信息提取出来。在提取信息的过程中可以采 用归一化层(Batch Normalisation)提高网络容量同时防止训练网络过程中出现梯度弥散。经过多个这样的基本单元结构，眼底图像中的特征可以被提取出来，最后经过全连接层和输出层(softmax)。
根据需要检测的眼底特征的类型，每个模块的网络层数在15到100层不等。具体的，卷积神经网络实现可以为如下结构：输入层-C1-BN1-R1-P1-C2-BN2-R2-P2-C3-BN3-R3-P3-C4-BN4-R4-P4–C5-BN5–R5–P5-FC1–FC2–SoftMax。其中输入层为一定尺寸大小的图像，C表示卷积层(同理C1、C2、C3、C4、C5)，BN表示批归一化层(同理BN1、BN2、BN3、BN4、BN5),R表示函数激活层(同理R1、R2、R3、R4、R5)，P表示池化层(同理P1、P2、P3、P4、P5)，全连接层为FC1和FC2，SoftMax提供输出。本实施例中所采用的卷积神经网络并不限于上述卷积神经网络的结构，其他能够满足本实施的神经网络结果同样适用。
由于第一特征的显著度大于第二特征和第三特征集的显著度，可以根据特征的显著度来改变神经网络的隐藏层大小，所称隐藏层是输入到输出之间的层。具体的，显著度大的特征采用较小的隐藏层，显著度小的特征采用较大的隐藏层。对于显著性小的第二特征和第三特征集的卷积网络的最大隐藏层大于第一特征的卷积网络的最大隐藏层。
具体地，在检测第一特征时，因为特征显著度大，需要网络的最大隐藏层尺寸较小，例如小于200x200，以便于提取特征。对于显著性小的第二特征或第三特征集，最大尺寸的隐藏层的输出应该保持较大，比如大于300x300,确保能够提取到细小的眼底子特征，例如细小的渗出点和出血点。 最大尺寸的隐藏层的输出的大小是由图像输入层、卷基层、池化层共同决定的，由多种方式实现，在这里不再赘述。
本发明实施例提供了一种基于机器学习的眼底图像检测装置，如图3所示，该检测装置包括：获取模块10，用于获取待检测的眼底图像；第一分类模型20，用于针对眼底图像的整体区域进行分类，确定眼底图像中是否包含第一特征；至少一个第二分类模型30，用于针对眼底图像中的特定区域进行分类，确定眼底图像中是否包含第二特征，其中第一特征的显著度大于第二特征的显著度；决策模块40，用于至少根据第一分类模型和第二分类模型的分类结果确定检测结果。
在可选的实施例中，特定区域和第二分类模型30分别为多个，多个第二分类模型分别用于针对不同的特定区域进行分类并输出第二分类结果，第二分类结果用于表示眼底图像中是否包含与特定区域相关的第二特征。
在可选的实施例中，第一分类模型和第二分类模型均为多分类模型，其分类结果用于表示眼底图像中是否包含第一特征和第二特征，以及第一特征和第二特征的具体类别。
在可选的实施例中，如图4所示，检测装置还包括：
第三分类模型50，用于针对眼底图像中的整体区域进行分类，确定眼底图像中是否包含第三特征，第三特征的显著度小于第二特征的显著度；
决策模块40根据第一分类模型20、第二分类模型30和第三分类模型50的分类结果确定检测结果。
在可选的实施例中，第三分类模型为多分类模型，其分类结果用于表示眼底图像中是否包含第三特征，以及第三特征的具体类别。
在可选的实施例中，特定区域包括视盘区域、黄斑区域、血管区域和视网膜区域中的至少一个区域。
在可选的实施例中，第一特征、第二特征和第三特征均为眼底病变特征。
一种电子设备，电子设备可以为服务器，也可以为终端。如图5所示，包括控制器，控制器包括一个或多个处理器41以及存储器42，图5中以一个处理器43为例。
电子设备还可以包括：输入装置43和输出装置44。
处理器41、存储器42、输入装置43和输出装置44可以通过总线或者其他方式连接，图4中以通过总线连接为例。
处理器41可以为中央处理器(Central Processing Unit，CPU)。处理器41还可以为其他通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现场可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等芯片，或者上述各类芯片的组合。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
存储器42作为一种非暂态计算机可读存储介质，可用于存储非暂态软件程序、非暂态计算机可执行程序以及模块。处理器41通过运行存储在存储器42中的非暂态软件程序、指令以及模块，从而执行服务器的各种功能应用以及数据处理，即实现上述方法实施例中的眼底图像检测方法。
存储器42可以包括存储程序区和存储数据区，其中，存储程序区可存储操作***、至少一个功能所需要的应用程序；存储数据区可存储根据服务器操作的处理装置的使用所创建的数据等。此外，存储器42可以包括高速随机存取存储器，还可以包括非暂态存储器，例如至少一个磁盘存储器件、闪存器件、或其他非暂态固态存储器件。在一些实施例中，存储器42可选包括相对于处理器41远程设置的存储器，这些远程存储器可以通过网络连接至网络连接装置。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
输入装置43可接收输入的数字或字符信息，以及产生与服务器的处理装置的用户设置以及功能控制有关的键信号输入。输出装置44可包括显示屏等显示设备。
一个或者多个模块存储在存储器42中，当被一个或者多个处理器41执行时，执行如图1或2所示的方法。
本发明实施例还提供了一种基于机器学***板或个人电脑。
本领域内的技术人员应明白，本发明的实施例可提供为方法、***、或计算机程序产品。因此，本发明可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本发明可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本发明是参照根据本发明实施例的方法、设备(***)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方 框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
显然，上述实施例仅仅是为清楚地说明所作的举例，而并非对实施方式的限定。对于所属领域的普通技术人员来说，在上述说明的基础上还可以做出其它不同形式的变化或变动。这里无需也无法对所有的实施方式予以穷举。而由此所引伸出的显而易见的变化或变动仍处于本发明创造的保护范围之中。

Claims (19)

  1. 一种基于机器学习的眼底图像检测方法，其特征在于，包括：
    获取待检测的眼底图像；
    利用第一分类模型针对所述眼底图像的整体区域进行分类，确定所述眼底图像中是否包含第一特征；以及利用至少一个第二分类模型针对所述眼底图像中的特定区域进行分类，确定所述眼底图像中是否包含第二特征，其中所述第一特征的显著度大于第二特征的显著度；
    至少根据所述第一分类模型和所述第二分类模型的分类结果确定检测结果。
  2. 根据权利要求1所述的方法，其特征在于，所述特定区域和所述第二分类模型分别为多个，在利用至少一个第二分类模型针对所述眼底图像中的特定区域进行分类的步骤中，分别利用不同的第二分类模型针对不同的特定区域进行分类并输出第二分类结果，所述第二分类结果用于表示所述眼底图像中是否包含与特定区域相关的第二特征。
  3. 根据权利要求1或2所述的方法，其特征在于，所述第一分类模型和所述第二分类模型均为多分类模型，其分类结果用于表示所述眼底图像中是否包含所述第一特征和所述第二特征，以及所述第一特征和所述第二特 征的具体类别。
  4. 根据权利要求1-3中任一项所述的方法，其特征在于，所述特定区域包括视盘区域、黄斑区域、血管区域和视网膜区域中的至少一个区域。
  5. 根据权利要求1-4中任一项所述的方法，其特征在于，所述第一特征和所述第二特征均为眼底病变特征。
  6. 根据权利要求1-5中任一项所述的方法，其特征在于，在确定检测结果之前，所述方法还包括：
    利用第三分类模型针对所述眼底图像中的整体区域进行分类，确定所述眼底图像中是否包含第三特征，所述第三特征的显著度小于所述第二特征的显著度；
    最终根据所述第一分类模型、所述第二分类模型和所述第三分类模型的分类结果确定检测结果。
  7. 根据权利要求6所述的方法，其特征在于，所述第三特征为眼底病变特征。
  8. 根据权利要求6所述的方法，其特征在于，所述第三分类模型为多分类模型，其分类结果用于表示所述眼底图像中是否包含所述第三特征，以及所述第三特征的具体类别。
  9. 一种电子设备，其特征在于，包括至少一个处理器；以及与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器执行包括以下内容的操作：
    获取待检测的眼底图像；利用第一分类模型针对所述眼底图像的整体区域进行分类，确定所述眼底图像中是否包含第一特征；以及利用至少一个第二分类模型针对所述眼底图像中的特定区域进行分类，确定所述眼底图像中是否包含第二特征，其中所述第一特征的显著度大于第二特征的显著度；至少根据所述第一分类模型和所述第二分类模型的分类结果确定检测结果。
  10. 根据权利要求9所述的设备，其特征在于，所述特定区域和所述第二分类模型分别为多个，所述处理器被配置为：在利用至少一个第二分类模型针对所述眼底图像中的特定区域进行分类的步骤中，分别利用不同的第二分类模型针对不同的特定区域进行分类并输出第二分类结果，所述第 二分类结果用于表示所述眼底图像中是否包含与特定区域相关的第二特征。
  11. 根据权利要求9或10所述的设备，其特征在于，所述第一分类模型和所述第二分类模型均为多分类模型，其分类结果用于表示所述眼底图像中是否包含所述第一特征和所述第二特征，以及所述第一特征和所述第二特征的具体类别。
  12. 根据权利要求9-11中任一项所述的设备，其特征在于，所述特定区域包括视盘区域、黄斑区域、血管区域和视网膜区域中的至少一个区域。
  13. 根据权利要求9-12中任一项所述的设备，其特征在于，所述第一特征和所述第二特征均为眼底病变特征。
  14. 根据权利要求9-13中任一项所述的设备，其特征在于，所述处理器被配置为：在确定检测结果之前，利用第三分类模型针对所述眼底图像中的整体区域进行分类，确定所述眼底图像中是否包含第三特征，所述第三特征的显著度小于所述第二特征的显著度；最终根据所述第一分类模型、所述第二分类模型和所述第三分类模型的分类结果确定检测结果。
  15. 根据权利要求14所述的设备，其特征在于，所述第三特征为眼底病变特征。
  16. 根据权利要求14所述的设备，其特征在于，所述第三分类模型为多分类模型，其分类结果用于表示所述眼底图像中是否包含所述第三特征，以及所述第三特征的具体类别。
  17. 一种计算机存储介质，其特征在于，其上存储有指令，当所述指令在计算机上运行时，使得所述计算机执行权利要求1-8中任意一项所述的眼底图像检测方法。
  18. 一种包含指令的计算机程序产品，其特征在于，当其在计算机上运行时，使得计算机执行权利要求1-8中任意一项所述的眼底图像检测方法。
  19. 一种基于机器学习的眼底图像检测***，其特征在于，包括：
    图像采集装置，用于采集眼底图像；
    根据权利要求9-16中任一项所述的设备，与所述图像采集装置通信，用于对所述眼底图像进行检测；
    输出装置，与所述眼底图像检测装置通信，用于输出眼底图像的检测结果。
PCT/CN2019/084210 2018-04-26 2019-04-25 基于机器学习的眼底图像检测方法、装置及*** WO2019206209A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP19792203.2A EP3785603B1 (en) 2018-04-26 2019-04-25 Machine learning-based fundus image detection method, apparatus, and system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810387484.8 2018-04-26
CN201810387484.8A CN108577803B (zh) 2018-04-26 2018-04-26 基于机器学习的眼底图像检测方法、装置及***

Publications (1)

Publication Number Publication Date
WO2019206209A1 true WO2019206209A1 (zh) 2019-10-31

Family

ID=63610363

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/084210 WO2019206209A1 (zh) 2018-04-26 2019-04-25 基于机器学习的眼底图像检测方法、装置及***

Country Status (3)

Country Link
EP (1) EP3785603B1 (zh)
CN (1) CN108577803B (zh)
WO (1) WO2019206209A1 (zh)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111291667A (zh) * 2020-01-22 2020-06-16 上海交通大学 细胞视野图的异常检测方法及存储介质
CN111696100A (zh) * 2020-06-17 2020-09-22 上海鹰瞳医疗科技有限公司 基于眼底影像确定吸烟程度的方法及设备
CN112905828A (zh) * 2021-03-18 2021-06-04 西北大学 一种结合显著特征的图像检索器、数据库及检索方法
CN113344894A (zh) * 2021-06-23 2021-09-03 依未科技(北京)有限公司 眼底豹纹斑特征提取及特征指数确定的方法和装置
CN113361487A (zh) * 2021-07-09 2021-09-07 无锡时代天使医疗器械科技有限公司 异物检测方法、装置、设备及计算机可读存储介质
CN113947124A (zh) * 2020-06-30 2022-01-18 中移(成都)信息通信科技有限公司 眼底彩色图像分类模型训练方法及眼底彩色图像分类方法
CN114494195A (zh) * 2022-01-26 2022-05-13 南通大学 用于眼底图像分类的小样本注意力机制并行孪生方法
CN117372284A (zh) * 2023-12-04 2024-01-09 江苏富翰医疗产业发展有限公司 眼底图像处理方法及***

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108577803B (zh) * 2018-04-26 2020-09-01 上海鹰瞳医疗科技有限公司 基于机器学习的眼底图像检测方法、装置及***
CN109528155B (zh) * 2018-11-19 2021-07-13 复旦大学附属眼耳鼻喉科医院 一种适用于高度近视并发开角型青光眼的智能筛查***及其建立方法
CN111374632B (zh) * 2018-12-29 2023-05-26 张国明 视网膜病变检测方法、装置及计算机可读存储介质
CN109859836B (zh) * 2019-01-15 2022-03-22 上海鹰瞳医疗科技有限公司 医疗图像识别方法和设备
CN109993731A (zh) * 2019-03-22 2019-07-09 依未科技(北京)有限公司 一种眼底病变分析方法及装置
CN110428421A (zh) * 2019-04-02 2019-11-08 上海鹰瞳医疗科技有限公司 黄斑图像区域分割方法和设备
CN109961848B (zh) * 2019-04-02 2022-03-22 北京鹰瞳科技发展股份有限公司 黄斑图像分类方法和设备
CN110021052B (zh) * 2019-04-11 2023-05-30 北京百度网讯科技有限公司 用于生成眼底图像生成模型的方法和装置
CN112190227B (zh) * 2020-10-14 2022-01-11 北京鹰瞳科技发展股份有限公司 眼底相机及其使用状态检测方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140314288A1 (en) * 2013-04-17 2014-10-23 Keshab K. Parhi Method and apparatus to detect lesions of diabetic retinopathy in fundus images
US9008391B1 (en) * 2013-10-22 2015-04-14 Eyenuk, Inc. Systems and methods for processing retinal images for screening of diseases or abnormalities
CN105411525A (zh) * 2015-11-10 2016-03-23 广州河谷互动医疗科技有限公司 一种眼底照片图像智能获取识别***
US9462945B1 (en) * 2013-04-22 2016-10-11 VisionQuest Biomedical LLC System and methods for automatic processing of digital retinal images in conjunction with an imaging device
CN108577803A (zh) * 2018-04-26 2018-09-28 上海鹰瞳医疗科技有限公司 基于机器学习的眼底图像检测方法、装置及***
CN108596895A (zh) * 2018-04-26 2018-09-28 上海鹰瞳医疗科技有限公司 基于机器学习的眼底图像检测方法、装置及***

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5387147B2 (ja) * 2009-06-03 2014-01-15 日本電気株式会社 病理画像診断システム、病理画像処理方法、病理画像診断プログラム
TWI578977B (zh) * 2011-04-07 2017-04-21 香港中文大學 視網膜圖像分析裝置
CN102722735A (zh) * 2012-05-24 2012-10-10 西南交通大学 一种融合全局和局部特征的内镜图像病变检测方法
EP3186779A4 (en) * 2014-08-25 2018-04-04 Agency For Science, Technology And Research (A*star) Methods and systems for assessing retinal images, and obtaining information from retinal images
CN104301585A (zh) * 2014-09-24 2015-01-21 南京邮电大学 一种运动场景中特定种类目标实时检测方法
CN107346409B (zh) * 2016-05-05 2019-12-17 华为技术有限公司 行人再识别方法和装置
CN107146231B (zh) * 2017-05-04 2020-08-07 季鑫 视网膜图像出血区域分割方法、装置和计算设备

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140314288A1 (en) * 2013-04-17 2014-10-23 Keshab K. Parhi Method and apparatus to detect lesions of diabetic retinopathy in fundus images
US9462945B1 (en) * 2013-04-22 2016-10-11 VisionQuest Biomedical LLC System and methods for automatic processing of digital retinal images in conjunction with an imaging device
US9008391B1 (en) * 2013-10-22 2015-04-14 Eyenuk, Inc. Systems and methods for processing retinal images for screening of diseases or abnormalities
CN105411525A (zh) * 2015-11-10 2016-03-23 广州河谷互动医疗科技有限公司 一种眼底照片图像智能获取识别***
CN108577803A (zh) * 2018-04-26 2018-09-28 上海鹰瞳医疗科技有限公司 基于机器学习的眼底图像检测方法、装置及***
CN108596895A (zh) * 2018-04-26 2018-09-28 上海鹰瞳医疗科技有限公司 基于机器学习的眼底图像检测方法、装置及***

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3785603A4

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111291667A (zh) * 2020-01-22 2020-06-16 上海交通大学 细胞视野图的异常检测方法及存储介质
CN111696100A (zh) * 2020-06-17 2020-09-22 上海鹰瞳医疗科技有限公司 基于眼底影像确定吸烟程度的方法及设备
CN113947124A (zh) * 2020-06-30 2022-01-18 中移(成都)信息通信科技有限公司 眼底彩色图像分类模型训练方法及眼底彩色图像分类方法
CN112905828A (zh) * 2021-03-18 2021-06-04 西北大学 一种结合显著特征的图像检索器、数据库及检索方法
CN113344894A (zh) * 2021-06-23 2021-09-03 依未科技(北京)有限公司 眼底豹纹斑特征提取及特征指数确定的方法和装置
CN113344894B (zh) * 2021-06-23 2024-05-14 依未科技(北京)有限公司 眼底豹纹斑特征提取及特征指数确定的方法和装置
CN113361487A (zh) * 2021-07-09 2021-09-07 无锡时代天使医疗器械科技有限公司 异物检测方法、装置、设备及计算机可读存储介质
CN114494195A (zh) * 2022-01-26 2022-05-13 南通大学 用于眼底图像分类的小样本注意力机制并行孪生方法
CN114494195B (zh) * 2022-01-26 2024-06-04 南通大学 用于眼底图像分类的小样本注意力机制并行孪生方法
CN117372284A (zh) * 2023-12-04 2024-01-09 江苏富翰医疗产业发展有限公司 眼底图像处理方法及***
CN117372284B (zh) * 2023-12-04 2024-02-23 江苏富翰医疗产业发展有限公司 眼底图像处理方法及***

Also Published As

Publication number Publication date
EP3785603B1 (en) 2023-08-02
CN108577803A (zh) 2018-09-28
EP3785603A1 (en) 2021-03-03
EP3785603A4 (en) 2021-06-23
CN108577803B (zh) 2020-09-01

Similar Documents

Publication Publication Date Title
WO2019206209A1 (zh) 基于机器学习的眼底图像检测方法、装置及***
WO2019206208A1 (zh) 基于机器学习的眼底图像检测方法、装置及***
CN106920219B (zh) 物品缺陷检测方法、图像处理***与计算机可读记录介质
CN108416344B (zh) 眼底彩照视盘与黄斑定位识别方法
Xu et al. FFU‐Net: Feature Fusion U‐Net for Lesion Segmentation of Diabetic Retinopathy
CN111325713A (zh) 基于神经网络的木材缺陷检测方法、***及存储介质
KR20200005409A (ko) 안저 이미지 관리 장치 및 안저 이미지의 적합성 판단 방법
CN109697719B (zh) 一种图像质量评估方法、装置及计算机可读存储介质
CN109961848B (zh) 黄斑图像分类方法和设备
KR20180065889A (ko) 타겟의 검측 방법 및 장치
CN110728681B (zh) 一种Mura缺陷检测方法及装置
US7643674B2 (en) Classification methods, classifier determination methods, classifiers, classifier determination devices, and articles of manufacture
CN111369523B (zh) 显微图像中细胞堆叠的检测方法、***、设备及介质
CN110956628B (zh) 图片等级分类方法、装置、计算机设备和存储介质
CN113706490B (zh) 一种晶片缺陷检测方法
US20220047159A1 (en) Glaucoma image recognition method and device and diagnosis system
CN106355584A (zh) 基于局部熵确定阈值的眼底图像微动脉瘤自动检测方法
KR20210012097A (ko) 딥러닝에 기반한 당뇨망막병증 검출 및 증증도 분류장치 및 그 방법
WO2019082202A1 (en) EARTHQUAKE IMAGE QUALITY ASSESSMENT SYSTEM
Sri et al. Diabetic retinopathy classification using deep learning technique
Ali et al. Optic Disc Localization in Retinal Fundus Images Based on You Only Look Once Network (YOLO).
TWI498830B (zh) 不均勻光線下的車牌辨識方法及系統
CN114175093A (zh) 显示面板的检测装置、检测方法、电子装置、可读介质
CN115861259A (zh) 一种基于模板匹配的引线框架表面缺陷检测方法及装置
CN116385717A (zh) 叶面病害识别方法、装置、电子设备、存储介质及产品

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19792203

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2019792203

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2019792203

Country of ref document: EP

Effective date: 20201126