CN114255234B

CN114255234B - Method for training model for identifying cardiovascular and cerebrovascular risks based on fundus images and related product

Info

Publication number: CN114255234B
Application number: CN202210190512.3A
Authority: CN
Inventors: 熊健皓; 赵昕; 和超; 张大磊
Original assignee: Beijing Yingtong Medical Technology Co ltd; Beijing Airdoc Technology Co Ltd
Current assignee: Beijing Yingtong Medical Technology Co ltd; Beijing Yingtong Smart Medical Technology Co Ltd; Beijing Airdoc Technology Co Ltd
Priority date: 2022-02-28
Filing date: 2022-02-28
Publication date: 2022-05-20
Anticipated expiration: 2042-02-28
Also published as: CN114255234A

Abstract

The invention provides a method for training a model for identifying cardiovascular and cerebrovascular risks based on fundus images and a related product thereof, wherein the method comprises the following steps: obtaining a period setting of a plurality of training periods for training of the model and a plurality of gradient constraint settings corresponding to the period setting, wherein the gradient constraint settings are applied to a plurality of output branches of the model, each output branch being associated with one of a plurality of indicators for determining cardiovascular risk and having a corresponding loss function; corresponding gradient constraint settings are applied during multiple trainings of a model training to update model parameters of the model based on the corresponding gradient constraint settings. By using the scheme of the invention, the recognition model can be effectively trained, the fundus image can be analyzed through the recognition model, and the noninvasive evaluation on cardiovascular and cerebrovascular risks can be realized. Therefore, compared with the prior art, the complexity of recognition is obviously simplified, and the accuracy of recognition is improved.

Description

Method for training model for identifying cardiovascular and cerebrovascular risks based on fundus images and related product

Technical Field

The present invention relates generally to the field of image analysis. More particularly, in one aspect, the invention relates to a method, a corresponding device and a computer-readable storage medium for training a model for identifying cardiovascular risk based on fundus images. In another aspect, the invention relates to a method of identifying cardiovascular risk based on a fundus image, a corresponding apparatus and a computer readable storage medium.

Background

Currently, there are a series of quantitative methods based on physical indicators for medically evaluating the cardiovascular and cerebrovascular risk degree. For example, various indexes such as age, sex, systolic pressure, diastolic pressure, etc. can be used to perform calculation by the risk function, so that a risk value, i.e., a probability, indicating that the evaluated person has a cardiovascular event within 5 to 10 years can be obtained. In order to identify the cardiovascular and cerebrovascular risk level of the evaluated person through image analysis, it is also proposed to perform an inference operation on an input fundus image by using a neural network model, so as to determine the cardiovascular and cerebrovascular risk level of the evaluated person only through the fundus image. Although the aforementioned identification of cardiovascular and cerebrovascular risks based on fundus images is easy to implement and convenient to assess, current identification schemes have drawbacks in terms of training and inference of such neural network models. For example, because of the difficulty of training without considering the indicators, the existing neural network model cannot achieve efficient training and thus the trained neural network model is also inefficient in inference. Therefore, how to overcome the recognition defects in the prior art and provide an efficient training method to realize accurate recognition becomes a technical problem which needs to be solved urgently.

Disclosure of Invention

In order to solve the technical problems, the invention provides a scheme for realizing noninvasive evaluation of cardiovascular and cerebrovascular risks based on fundus images. In particular, the present invention enables a non-invasive and effective assessment of cardiovascular and cerebrovascular risk by training a neural network with multiple branches. To this end, the present invention provides solutions in a number of aspects as follows.

In a first aspect, the invention provides a method for training a model for identifying cardiovascular and cerebrovascular risks based on a fundus image, the method being implemented by a computing device and comprising: obtaining a period setting for a plurality of training periods trained on the model and a plurality of gradient constraint settings corresponding to the period setting, wherein the gradient constraint settings are applied to a plurality of output branches of the model, each output branch of the plurality of output branches being associated with one of a plurality of indicators for determining cardiovascular risk and having a corresponding loss function; applying corresponding gradient constraint settings in a reverse training of a plurality of training periods of the model training to update model parameters of the model based on the corresponding gradient constraint settings until a training operation on the model is completed.

In one embodiment, the gradient constraint setting is associated with a corresponding training difficulty of a plurality of indexes during the training.

In one embodiment, the plurality of training periods includes a first training period and a second training period, the method comprising: applying, during a first training of the model, a first gradient constraint on a plurality of output branches of the model to update model parameters of the model based on the first gradient constraint in a reverse training of the model; and applying a second gradient constraint to a plurality of output branches of the model during a second training period subsequent to the first training period to update the model parameters based on the second gradient constraint in the reverse training of the model.

In one embodiment, wherein in the inverse training of a model updating model parameters of the model based on the first and second gradient constraints, the method comprises: updating model parameters of the model based on the first gradient constraint so that indicators with low training difficulty have model parameters with high weights and indicators with high training difficulty have model parameters with low weights; and updating model parameters of the model based on the second gradient constraint so that the indicators with low training difficulty have model parameters with decreasing weights and the indicators with high training difficulty have model parameters with increasing weights.

In one embodiment, the plurality of indicators includes a plurality of conventional indicators associated with the cardiovascular risk, and a risk function indicator based on the plurality of conventional indicators, wherein the risk function indicator value indicates a cardiovascular risk level associated with the fundus image.

In one embodiment, the plurality of general indicators includes a plurality of items selected from age, gender, systolic blood pressure, diabetes history, total cholesterol, body mass index, and the risk function includes a cox function, a logistic regression-based risk function, a cox function-modified risk function, or a logistic regression-modified risk function, wherein the age and gender are indicators of low difficulty in training, and the systolic blood pressure, diabetes history, total cholesterol, and body mass index are indicators of high difficulty in training.

In one embodiment, during the first training, a first gradient constraint is used as follows:

wherein g _ Age represents the gradient of an Age output branch, g _ gentr represents the gradient of a sex output branch, g _ SBP represents the gradient of a systolic blood pressure output branch, g _ DB represents the gradient of a diabetes history output branch, g _ TC represents the gradient of a total cholesterol output branch, and g _ BMI represents the gradient of a body mass indicator output branch.

In one embodiment, during the second training, a second gradient constraint is used as follows:

wherein

And the gradient of any output branch in age, sex, systolic blood pressure, diabetes history, total cholesterol and body quality index is shown.

In a second aspect, the present invention provides a method of identifying cardiovascular risk based on a fundus image, comprising: acquiring a fundus image to be identified; inputting the fundus image into a model trained according to the method in the first aspect and embodiments thereof; and determining a cardiovascular risk level reflected by the fundus image from the output of the model.

In a third aspect, the invention provides an apparatus for training a model for identifying cardiovascular risk based on a fundus image, comprising: a processor; and a memory storing program instructions for training a model for identifying cardiovascular and cerebrovascular risk based on a fundus image, which when executed by the processor, cause the apparatus to implement the method described in the first aspect and embodiments thereof.

In a fourth aspect, the invention provides a computer readable storage medium storing program instructions for training a model for identifying cardiovascular and cerebrovascular risks based on a fundus image, which when executed by a processor, cause the method described in the first aspect and its various embodiments to be carried out.

In a fifth aspect, the present invention provides an apparatus for identifying cardiovascular and cerebrovascular risks based on fundus images, characterized by comprising: a processor; and a memory storing program instructions for identifying cardiovascular risk based on a fundus image, which when executed by the processor, cause the apparatus to implement the method described in the second aspect.

In a sixth aspect, the invention provides a computer readable storage medium storing program instructions for identifying a cardiovascular risk based on a fundus image, which when executed by a processor, causes the method described in the second aspect to be carried out.

With the solutions described in the above aspects and embodiments thereof, the present invention can achieve noninvasive evaluation of cardiovascular and cerebrovascular risks simply by analyzing fundus images, thereby significantly simplifying the complexity of evaluation relative to the prior art. Further, by respectively establishing the loss functions by using the model branches aiming at different indexes, the overall gradient constraint aiming at each branch can be realized, and thus the model weights of different indexes can be timely adjusted. Through the gradient constraint mechanism, the matching between the training process of the model and the learning difficulty of the index can be realized, so that the training efficiency of the model and the identification accuracy of the risk are improved. Therefore, by the model training and identifying method, a high-quality risk prediction model can be obtained, and accurate risk prediction and evaluation can be provided.

Drawings

The above and other objects, features and advantages of exemplary embodiments of the present invention will become readily apparent from the following detailed description read in conjunction with the accompanying drawings. In the accompanying drawings, several embodiments of the present invention are illustrated by way of example and not by way of limitation, and like reference numerals designate like or corresponding parts throughout the several views, in which:

FIG. 1 is a simplified flow diagram illustrating a method for training a model for identifying cardiovascular risk based on fundus images, according to an embodiment of the present invention;

FIG. 2 is a simplified flow diagram illustrating a method for identifying cardiovascular risk based on fundus images according to an embodiment of the present invention;

FIG. 3 is a block diagram illustrating a schematic network architecture of a model for identifying cardiovascular risk based on fundus images, according to an embodiment of the present invention;

FIG. 4 is a schematic diagram showing more detail of the network of FIG. 3 during reverse training;

FIG. 5 is a detailed flow diagram illustrating a method for training a model for identifying cardiovascular risk based on fundus images, according to an embodiment of the present invention;

FIG. 6 is a graph showing the recognition results of a prior art model;

FIG. 7 is a graph showing the recognition results of a model obtained using a scheme according to an embodiment of the invention;

FIG. 8 is a schematic diagram illustrating an apparatus for model training and/or cardiovascular risk identification according to an embodiment of the invention;

FIG. 9 is a block diagram illustrating a system for model training and/or cardiovascular risk identification according to an embodiment of the invention.

Detailed Description

Aspects of the present invention and their corresponding embodiments will now be described with reference to the accompanying drawings. It will be appreciated that for simplicity and clarity of illustration, where considered appropriate, reference numerals may be repeated among the figures to indicate corresponding or analogous elements. In addition, numerous specific details are set forth in order to provide a thorough understanding of the embodiments described herein. However, based on the disclosure and teachings of the present invention, one of ordinary skill in the art may practice the embodiments described herein without these specific details. In other instances, well-known methods, procedures, and components have not been described in detail so as not to obscure the aspects of the present invention. Additionally, the following description of the present invention is intended to be a full and exemplary description of the invention, and should not be taken to limit the scope of the embodiments described herein.

In order to realize effective identification or evaluation of cardiovascular and cerebrovascular risks, the scheme of the invention provides that noninvasive evaluation of cardiovascular and cerebrovascular risks is realized based on fundus images. Specifically, the scheme of the invention predicts the input target fundus image based on the risk prediction model of the cardiovascular and cerebrovascular vessels, thereby avoiding measuring and calculating indexes related to the cardiovascular and cerebrovascular vessel risk and obviously simplifying the risk evaluation process. In one embodiment, the model for cardiovascular and cerebrovascular risk prediction according to the invention can be a model constructed based on a deep learning neural network model, and can be trained by using output branches corresponding to a plurality of indexes. Further, the invention can improve the training efficiency by setting a gradient constraint mechanism in the reverse training of the model, and finally obtain the model capable of performing inference (namely evaluation or prediction).

The aspects of the present invention will be described in detail below with reference to the accompanying drawings.

Fig. 1 is a simplified flow diagram illustrating a method 100 for training a model for identifying cardiovascular risk based on fundus images, in accordance with an embodiment of the present invention. Based on the description below, one skilled in the art will appreciate that the method 100 herein may be performed by a computing device. According to various embodiments, the computing device may be a general purpose computing device including a processor and a memory, or an artificial intelligence device including a special purpose processor (e.g., an artificial intelligence processor) and a memory. Further, the fundus image may be an image captured by a fundus camera for acquiring a fundus retinal image.

As shown in fig. 1, at step S102, period settings of a plurality of training periods for the model training and a plurality of gradient constraint settings corresponding to the period settings are acquired. In order to match the training process of the model with the learning difficulty of the labels (i.e. the multiple indicators of the invention for assessing cardiovascular risk), the invention proposes to set multiple (e.g. greater than or equal to two) training periods, i.e. period settings, in the training process. As an exemplary embodiment, two training periods may be provided, namely a first training period and a second training period in the context of the present invention. Further, for each training period, the invention proposes to set a corresponding gradient constraint set in association with the training difficulty of a plurality of indexes during the corresponding training period.

Based on the above period settings and gradient constraint settings, at step S104, corresponding gradient constraint settings are applied in the inverse training of the above training periods of the model, so as to update the model parameters of the model based on the corresponding gradient constraint settings until the training operation on the model is completed. Specifically, by setting the foregoing gradient constraints and applying the gradient constraints to a plurality of output branches of the model, such as a plurality of branches of age, sex, systolic blood pressure ("SBP") and the like shown in fig. 3 and 4, it is possible to adjust model parameters (e.g., weights) of the respective branches in the reverse training. To achieve such an adjustment, the present invention proposes, as an example, to set a corresponding penalty function for each branch, whereby the gradient of each branch can be obtained in the training. Based on this, corresponding different gradient constraints can be implemented during different training periods to constrain the gradient values of the respective branches, so that corresponding model parameters, such as weights, can be appropriately adjusted according to the learning difficulty of the labels in the reverse training.

In one embodiment, the index for evaluating cardiovascular and cerebrovascular risks of the invention may include a plurality of conventional indexes associated with the cardiovascular and cerebrovascular risks, and a risk function index based on the plurality of conventional indexes, wherein the risk function index value indicates a cardiovascular and cerebrovascular risk level associated with the fundus image. Further, according to a specific embodiment, the aforementioned plurality of general indicators may include a plurality of items of Age ("Age"), Gender ("Gender"), systolic blood pressure ("SBP"), diabetes history ("DB"), total cholesterol ("TC"), body mass index ("BMI"), and the risk function includes a Cox function, a logistic regression-based risk function, a Cox function-modified risk function, or a logistic regression-based risk function-modified risk function, wherein the Age and Gender are indicators of low difficulty in training, and the systolic blood pressure, diabetes history, total cholesterol, and body mass index are indicators of high difficulty in training.

As an example, the Cox function can be expressed as the following equation (1):

（1）

wherein

Is a weighted ratio of the respective indices. X is an index value (such as age),

is the mean of each index used for model building. Each index item may be used with a dimensional treatment, such as the age of the model being replaced by ln (age) and the gender being replaced by 0/1 (1 and 0 for male and female respectively). Assume age and gender and their corresponding weights

And

24.87 and 0.36, respectively, and the mean age of the population modeled for use

And sex

39.5 and 0.49, then a simple age and gender based risk calculation is established as in equation (2) below:

（2）

wherein

Is constant and may take the value 0.9707, while the output Risk value is typically between 0-1.

For a risk function based on logistic regression, it can be expressed by modeling as the following formula (3):

（3）

it is calculated in a similar manner to the Cox function, where β is the weighted value of each index,Xis the value of the index,

is a constant value, usually referred to as the intersection value of the function curve with the coordinates.

Considering that the risk value samples obtained by calculating the risk model corresponding to the Cox function and the logistic regression model have highly uneven numerical value distribution and obvious order difference between numerical values, the invention also provides the improved risk function based on the Cox function or the logistic regression function. As an example, the improved risk function may be expressed in equation (4) below:

（4）

the formula (4) is based on eRiskLogarithm of function value, whereinRisk’I.e. to improve the risk formula later,Riskmay have expression forms as shown in equations (2) and (3). When the model is trained using the improved risk function, the risk values are distributed more evenly and numerically more linearly over the whole, and the maximum and minimum values generally do not exceed a numerical difference of an order of magnitude. Further, using the improved risk letterWhen the model is trained, the model can be converged more quickly. In addition, compared with the original risk function, the improved risk function can keep the correspondence of the original sample risk sequence, and the improved sample value distribution is beneficial to the feature learning of the deep learning model.

Fig. 2 is a simplified flow diagram illustrating a method 200 for identifying cardiovascular risk based on fundus images according to an embodiment of the present invention. Similar to the methods previously used to train models, the method 200 may be implemented by a general purpose computing device or a special purpose computing device. In one implementation scenario, method 200 may also be performed by a fundus camera integrated with the aforementioned computing device.

As shown in fig. 2, at step S202, a fundus image to be recognized is acquired. Specifically, the fundus image may be a fundus image taken by the person under evaluation via a fundus camera. Next, at step S204, the fundus image is input into a trained model, that is, the model of the present invention as described above in connection with fig. 1, to perform an inference operation for the fundus image. Finally, at step S206, the cardiovascular risk level reflected by the fundus image is determined from the output of the aforementioned model. In one embodiment, the output of the aforementioned model may be a specific risk value that may reflect the probability of a cardiovascular event occurring within the next 5-10 years of the person being evaluated.

Fig. 3 is a schematic network configuration block diagram showing a model for identifying cardiovascular and cerebrovascular risks based on fundus images according to an embodiment of the present invention. Further, fig. 4 shows more details of the network in fig. 3 during reverse training.

As previously mentioned, the model of the present invention may be built based on a deep learning neural network model, and the backbone network as shown in the figure may be an initiation-Resnet-v 2 based network. After the fundus image (e.g., of size 299 x 299) is input into the backbone network, the backbone network may return a multi-dimensional vector, such as an 8 x 1536 vector. Additionally or alternatively, the multidimensional vector may be processed, for example, via a pooling layer (not shown) to reduce data dimensionality, for example, via an averaging pooling operation. Then, optionally, dropout operation in the deep learning neural network may be performed on the result output after the pooling operation of the pooling layer, so as to prevent overfitting problem occurring in the training process, and weaken joint adaptability among the neural nodes in the network, thereby enhancing generalization capability.

After that, the output data after dropout processing can be input to a full connection unit (i.e. 1 × 32 dense in the figure) with the size of 1 × 32 as shown in the figure, and the full connection unit is connected to a single full connection unit (i.e. 1 × 7 dense in the figure), so that the predicted values of the present invention in the model training stage, such as the predicted values of the indexes shown in the figure, can be output. As shown in the figure and as previously described, the model of the invention has a plurality of output branches for a plurality of indicators for identifying cardiovascular risk. Specifically, an age output branch, a gender output branch, an SBP output branch, a diabetes history (i.e., DB with a diabetes value of 1 and no diabetes value of 0), a Total Cholesterol (TC) output branch, a BMI output branch, and a risk function output branch as exemplarily shown in the figure may be set. It is to be understood that the Cox risk function output branches shown in the figures are merely exemplary and not limiting, and that the risk function may also be a logistic regression-based risk function as described above, or a Cox-based risk function and a logistic regression-based risk function modified risk function.

In order to achieve an efficient update of the model parameters, the invention proposes to set a loss function for each output branch (or for each index), as described above. In other words, each output branch of the present invention has its associated loss function. As an example, the penalty function for each output branch may be set in the following manner. When the tag value of the output branch (i.e., the amount of an indicator in the training data set that is known to have been measured in advance or calculated for the risk function) is less than or equal to a certain threshold, then the loss function is set to the absolute value of the difference between the tag value and the training value (i.e., the predicted value or the estimated value of the risk level). In contrast, when the tag value of the output branch is greater than the certain threshold, the penalty function is set to the minimum value between the absolute value and the preset penalty value. Here, the preset loss value may be a preset parameter for controlling the contribution of the training sample data to the loss function this time.

As an example, the loss function of each branch of the model may also be set as a Mean Square Error (MSE) and a Mean Absolute Error (MAE), where MSE and MAE have the following expressions, respectively:

（5）

(6)

whereinY _iA true measured value representing a certain index,

is the metric value of the model prediction (i.e., the predicted value of the branch), and n is the number of samples.

After the loss function is determined, the deep learning neural network model of the present invention can be trained, which includes forward training and reverse training. Specifically, on the forward training, a fundus image for training, for example, a fundus image of the size 299 × 299, may be input to the deep learning neural network, and a prediction or evaluation result of the forward training, that is, the above-mentioned training value or prediction value, is obtained. Next, a gradient descent algorithm, for example, in the reverse direction is performed to update parameters (e.g., weights or biases) of the deep-learning neural network model based on the above loss function determined by the training values and the tag values of the current round. Through a plurality of rounds of forward and backward training, and optionally verification and testing through the verification set and the test set, a deep learning neural network model capable of performing inference, namely the cardiovascular risk prediction model can be finally obtained.

With respect to the above-mentioned inverse training, as mentioned above, the present invention proposes to set different gradient constraints during different training periods to achieve gradient adjustment in the inverse training of different output branches and thus adjust the corresponding model parameters. As an example, when the training period of the model is set to two training periods, i.e., the first training period and the second training period, the present invention proposes the following gradient constraint.

Specifically, for the first training period, the present invention proposes a gradient constraint expressed using the following equations (5) and (6):

（5）

（6）

wherein here

And

i.e. the gradient of the respective output branches in the inverse training as shown in fig. 4. Through the gradient constraint expressed by the formulas (5) and (6), the indexes (such as the indexes with low training difficulty) in the weight updating operation of the reverse training can be realizedAgeAndGender) Indicators with high weighted model parameters and high training difficulty (e.g.SBP、DB、TCAndBMI) Model parameters with low weights. In some scenarios, the gradient of the index that is harder to learn can be constrained during early training by the gradient constraint mechanism of the present invention such that the weight of its corresponding model parameter is very low, even up to 0 weight.

During a second training period, temporally subsequent to the first training period, the present invention proposes a gradient constraint expressed by the following formula (7):

（7）

whereiniRepresents any of the branches Age, genter, DB, TC, BMI and SBP. Through the gradient constraint expressed by the formula (7), the indexes with low training difficulty (such as the indexes with low training difficulty) can be obtained in the weight updating operation of the reverse trainingAgeAndGender) Indicators with model parameters having decreasing weights and that make training difficult (e.g., such asSBP、DB、TCAndBMI) Model parameters with rising weights. By setting and gradient constraint mechanisms during training, the method can lead the model to be fully learned at the initial stage or to be trained with easily learned indexes, and lead the model to be learned with difficultly learned indexes when the performance of the loss function of the model tends to be stable at the later stage of training. Therefore, huge performance fluctuation of the model during initial training can be avoided, and the model can be better converged.

Fig. 5 is a detailed flow diagram illustrating a method 500 for training a model for identifying cardiovascular risk based on fundus images, according to an embodiment of the present invention. Based on the following description, those skilled in the art will appreciate that the method 500 may be regarded as a specific implementation of the method 200 described above in conjunction with fig. 2. Further, the model referred to in fig. 5 may be a model as illustrated in fig. 3 and 4. The foregoing description with respect to fig. 2-4 therefore applies equally to method 500. For the sake of brevity, the same will not be described in detail below.

As shown in fig. 5, at step S502, the period settings and gradient constraint settings for model training are acquired. As previously described, the present invention divides the model training phase to obtain a plurality of training sessions. In one embodiment, the partitioning may be based on a duration or number of training iterations. Taking the number of iterations of training as a reference, setting the period as two training periods, the first N epochs may be set as the first training period, and the subsequent M epochs until the training is completed may be set as the second training period. Here, an epoch means that the model completes one iteration for all fundus images of a given training set. In one implementation scenario, where the model of the present invention is trained through 10-80 iterations (i.e., 10-80 epochs), the aforementioned N corresponding to the first training period may range from 10-25, and the aforementioned M corresponding to the second training period may range from 55-70.

Regarding gradient constraint setting, as described above, the present invention proposes to set different gradient constraints for different training periods so as to appropriately adjust model parameters according to the difficulty of label learning. Based on this, at step S504, a first gradient constraint is applied during the first training. The first gradient constraint as expressed by equations (5) and (6) may be applied for the first training period, which is an early stage of training, in view of the need to give higher weight to the easy-to-learn label at this time to avoid excessive fluctuation in the model output performance. Next, model parameters, e.g., corresponding weights, are updated in a backward training based on the first gradient constraint. Similarly, at step S508, a second gradient constraint is applied during a second training period that is temporally subsequent to the first training period. For example, a gradient constraint as expressed by equation (7) may be applied. Next, at step S510, model parameters are updated in a reverse training based on the second gradient constraint.

For the case of dividing the training period into N (N is an integer greater than 2) training periods, applying the nth gradient constraint during the nth training period may be performed at step S512 and updating the model parameters in the inverse training based on the nth gradient constraint may be performed at step S514. Finally, at step S516, the training phase of the model is completed by, for example, validation of the validation set.

Fig. 6 and 7 show graphs of recognition results of a prior art model and a model obtained using a scheme according to an embodiment of the present invention, respectively. The identification result was based on 4988 random samples, and 485 samples with risk greater than 5% were obtained by calculating the Cox risk function. After the 485 samples were set as positive and the remaining 4503 samples as negative, when the label values were calculated directly using the Cox risk function and model training was performed with the mean square error function ("MSE") as the corresponding loss function, the model results had a prediction result as shown in fig. 6, i.e., the two-class AUC ("Area Under Curve" for cardiovascular risk) was 0.865. In contrast, when the model structure of the present invention is used and the risk function is a Cox risk function, then the model results as shown in fig. 7 can be obtained, i.e., the two-class AUC of cardiovascular risk is 0.926. In particular, the specificity (x axis in the figure) and the sensitivity (y axis in the figure) are respectively 0.791 and 0.909, i.e. the prediction model of the model has a significant distinguishing effect on both negative samples and positive samples. Therefore, the cardiovascular and cerebrovascular risk prediction model based on the scheme of the invention can be used for large-scale screening of cardiovascular and cerebrovascular diseases.

Fig. 8 is a diagram illustrating a device 800 for model training and/or cardiovascular risk identification according to an embodiment of the invention. As shown in fig. 8, device 800 includes, among other things, a processor 801 and a memory 802. In accordance with aspects of the present invention, memory 802 stores program instructions for training a model for identifying cardiovascular risk based on a fundus image and/or program instructions for identifying cardiovascular risk based on a fundus image. When the processor 801 executes the program instructions, it may implement the training method described in conjunction with fig. 1 and 3 or the recognition method described in conjunction with fig. 2, so as to output the recognition result as shown in the figure, i.e. the risk level of the evaluated person to suffer from cardiovascular and cerebrovascular diseases.

Fig. 9 is a block diagram illustrating a system 900 for identifying cardiovascular risk in accordance with an embodiment of the present invention. As shown in the figure, the system 900 may include a device 901 and its peripheral devices and external networks according to embodiments of the present invention, wherein the device 901 may include a device as described above in connection with fig. 8 and may be used to perform the aspects of the present invention discussed in connection with fig. 1-7, including but not limited to acquiring a fundus image to be predicted or evaluated, training a prediction model for identifying cardiovascular risk, and performing an inference operation regarding cardiovascular risk level using the prediction model.

As shown in fig. 9, the device 901 of the present invention may include a CPU911, which may be a general purpose CPU, a dedicated CPU (such as a dedicated graphics processor GPU), or other execution unit for processing and executing information. Further, the device 901 may further include a mass memory 912 and a read only memory ROM 913, wherein the mass memory 912 may be configured to store various types of data including training data based on fundus images, intermediate data, training results, and various programs required to run a model of the present invention such as a deep learning neural network model, and the ROM 913 may be configured to store a power-on self-test for the device 901, initialization of various functional modules in the system, a driver of basic input/output of the system, and data required to boot the operating system.

Further, the device 901 may also include other hardware platforms or components, such as the illustrated Tensor Processing Unit (TPU) 914, Graphics Processing Unit (GPU) 915, Field Programmable Gate Array (FPGA) 916, and Machine Learning Unit (MLU) 917. It is understood that although various hardware platforms or components are shown in the device 901, this is by way of illustration and not by way of limitation, and those skilled in the art may add or remove corresponding hardware as may be desired. For example, the device 901 may also include only a CPU for performing respective model training, optimization and inference operations.

To enable the transfer of information, the device 901 of the present invention further includes a communication interface 918 through which it may connect to a local area network/wireless local area network (LAN/WLAN) 905, which in turn may connect to a local server 906 or to the Internet ("Internet") 907 through the LAN/WLAN. Alternatively or additionally, device 901 of the present invention may also be connected directly to the internet or a cellular network via communication interface 918 based on wireless communication technology, such as third generation ("3G"), fourth generation ("4G"), or 5 generation ("5G") based wireless communication technology. In some application scenarios, the device 901 of the present invention may also access a server 908 of an external network and possibly a database 909 as needed in order to obtain various known neural network models, data and modules, and may remotely store various data used or generated in training and inference.

Additionally or alternatively, when the apparatus 901 of the present invention is implemented as a component in a fundus camera, then the peripheral devices thereof may further include a plurality of motors 902, an imaging device 903, and an input/display device 904. In one embodiment, the motor 902 may be a stepper motor that moves the main camera, while the imaging device includes the main camera and a binocular system fixedly connected to the main camera, which is composed of two sub-cameras. In one embodiment, the input devices in input/display device 904 include, for example, a keyboard, mouse, microphone, or other input buttons or controls configured to receive user instructions, while the display device may include, for example, one or more speakers and/or one or more visual or touch-sensitive displays configured to audibly prompt and/or visually display the training or inferred results.

The above-mentioned CPU911, mass storage 912, ROM 913, TPU 914, GPU 915, FPGA 916, MLU 917 and communication interface 918 of the device 501 of the present invention may be interconnected by a bus 919, and enable data interaction with peripheral devices through the bus. Through the bus 919, the CPU911 may control other hardware components and their peripherals in the apparatus 901, in one embodiment.

The device of the invention can be flexibly arranged according to different application scenes. In one implementation scenario, the device of the present invention may be arranged at a cloud server for receiving a fundus image from a local end device and feeding back a cardiovascular risk level reflected by the fundus image to the local end device. In another implementation scenario, the device of the present invention may be disposed at a local end device for receiving a fundus image and assessing a cardiovascular risk level reflected by the fundus image.

It should also be understood that while aspects of the present invention may also be implemented via computer instructions, the computer instructions may be stored on a computer-readable medium. According to various implementations, the computer-readable medium, such as a storage medium, computer storage medium, or data storage device (removable) and/or non-removable) such as, for example, a magnetic disk, optical disk, or magnetic tape. Computer storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules or other data.

Based on the above, the present invention also discloses a computer readable storage medium having stored therein program instructions adapted to be loaded by a processor and to cause the inventive apparatus to perform the inventive training and recognition scheme as described above in connection with fig. 1-7.

The computer readable storage medium may be any suitable magnetic or magneto-optical storage medium, such as resistive Random Access Memory (rram), Dynamic Random Access Memory (dram), Static Random Access Memory (SRAM), enhanced Dynamic Random Access Memory (edram), High-Bandwidth Memory (HBM), hybrid Memory cubic (hmc) Memory cube, and the like, or any other medium that can be used to store the desired information and that can be accessed by an application, a module, or both. Any such computer storage media may be part of, or accessible or connectable to, the apparatus of the invention. Any of the applications or modules described herein may be implemented using computer-readable/executable instructions that may be stored or otherwise maintained by such computer-readable media.

It should be understood that the possible terms "first" or "second" etc. in the claims, the description and the drawings of the present disclosure are used for distinguishing between different objects and not for describing a particular order. The terms "comprises" and "comprising," when used in the specification and claims of this disclosure, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only, and is not intended to be limiting of the invention disclosed. As used in the specification and claims of this disclosure, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should be further understood that the term "and/or" as used in this disclosure and claims refers to any and all possible combinations of one or more of the associated listed items and includes such combinations.

Although the embodiments of the present invention are described above, the descriptions are only examples adopted for understanding the present invention, and are not intended to limit the scope and application scenarios of the present invention. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

1. A method for training a model for identifying cardiovascular risk based on fundus images, the method being implemented by a computing device and comprising:

obtaining a period setting for a plurality of training periods trained on the model and a plurality of gradient constraint settings corresponding to the period setting, wherein the gradient constraint settings are applied to a plurality of output branches of the model, each output branch of the plurality of output branches being associated with one of a plurality of indicators for determining cardiovascular risk and having a corresponding loss function;

applying corresponding gradient constraint settings in a reverse training of a plurality of training periods of the model training to update model parameters of the model based on the corresponding gradient constraint settings until a training operation on the model is completed,

wherein the gradient constraint setting is associated with a corresponding training difficulty of a plurality of indicators during the training;

the plurality of training periods includes a first training period and a second training period, the method comprising:

applying, during a first training of the model, a first gradient constraint on a plurality of output branches of the model to update model parameters of the model based on the first gradient constraint in a reverse training of the model; and

applying a second gradient constraint to a plurality of output branches of the model during a second training period subsequent to the first training period to update the model parameters based on the second gradient constraint in the reverse training of the model.

2. The method of claim 1, wherein in updating model parameters of a model based on the first and second gradient constraints in a reverse training of the model, the method comprises:

updating model parameters of the model based on the first gradient constraint such that indicators of low training difficulty have model parameters of high weight and indicators of high training difficulty have model parameters of low weight; and

updating model parameters of the model based on the second gradient constraint such that indicators of low training difficulty have model parameters of decreasing weight and indicators of high training difficulty have model parameters of increasing weight.

3. The method according to claim 1 or 2, characterized in that the plurality of indices comprise a plurality of conventional indices associated with the cardiovascular risk and a risk function index based on the aforementioned plurality of conventional indices, wherein a risk function index value indicates a cardiovascular risk level associated with the fundus image.

4. The method of claim 3, wherein the plurality of conventional indicators includes a plurality of indicators selected from age, gender, systolic blood pressure, history of diabetes, total cholesterol, body mass index, and the risk function includes a cox function, a logistic regression-based risk function, a cox function-modified risk function, or a logistic regression-based improved risk function, wherein the age and gender are indicators of low difficulty in training and the systolic blood pressure, history of diabetes, total cholesterol, and body mass index are indicators of high difficulty in training.

5. The method of claim 4, wherein during the first training, a first gradient constraint is used as follows:

wherein

The gradient of the age output branch is represented,

a gradient representing a sex output branch,

A gradient representing the systolic pressure output branch,

The gradient of the output branch of the diabetes history,

A gradient showing a total cholesterol output branch,

Representing the gradient of the body mass indicating output branch.

6. The method of claim 5, wherein during the second training, a second gradient constraint is used as follows:

wherein

7. A method for identifying cardiovascular and cerebrovascular risks based on fundus images, comprising:

acquiring a fundus image to be identified;

inputting the fundus image into a model trained according to any one of the methods of claims 1-6; and

determining a cardiovascular risk level reflected by the fundus image from an output of the model.

8. An apparatus for training a model for identifying cardiovascular risk based on fundus images, comprising:

a processor; and

a memory storing program instructions for training a model for identifying cardiovascular risk based on fundus images, which when executed by the processor, cause the apparatus to implement the method of any of claims 1-6.

9. A computer readable storage medium storing program instructions for training a model for identifying cardiovascular risk based on fundus images, which when executed by a processor, cause the method according to any one of claims 1-6 to be carried out.

10. An apparatus for identifying cardiovascular risk based on fundus images, comprising:

a processor; and

a memory storing program instructions to identify cardiovascular risk based on a fundus image, which when executed by the processor, cause the apparatus to implement the method of claim 7.

11. A computer readable storage medium storing program instructions for identifying cardiovascular risk based on a fundus image, which when executed by a processor, cause the method of claim 7 to be carried out.