Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning

Abstract

Traditionally, medical discoveries are made by observing associations, making hypotheses from them and then designing and running experiments to test the hypotheses. However, with medical images, observing and quantifying associations can often be difficult because of the wide variety of features, patterns, colours, values and shapes that are present in real data. Here, we show that deep learning can extract new knowledge from retinal fundus images. Using deep-learning models trained on data from 284,335 patients and validated on two independent datasets of 12,026 and 999 patients, we predicted cardiovascular risk factors not previously thought to be present or quantifiable in retinal images, such as age (mean absolute error within 3.26 years), gender (area under the receiver operating characteristic curve (AUC) = 0.97), smoking status (AUC = 0.71), systolic blood pressure (mean absolute error within 11.23 mmHg) and major adverse cardiac events (AUC = 0.70). We also show that the trained deep-learning models used anatomical features, such as the optic disc or blood vessels, to generate each prediction.

This is a preview of subscription content, access via your institution

Access options

Buy this article

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Predictions of age and SBP.
Fig. 2: Attention maps for a single retinal fundus image.

Similar content being viewed by others

References

  1. WHO The Top 10 Causes of Death (2017).

  2. Stone, N. J. et al. 2013 ACC/AHA guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk in adults: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation 129, S1–S45 (2014).

    Article  PubMed  Google Scholar 

  3. Wilson, P. W. et al. Prediction of coronary heart disease using risk factor categories. Circulation 97, 1837–1847 (1998).

    Article  CAS  PubMed  Google Scholar 

  4. National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III). Third Report of the National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III) final report. Circulation 106, 3143–3421 (2002).

    Google Scholar 

  5. D’Agostino, R. B. Sr et al. General cardiovascular risk profile for use in primary care: the Framingham Heart Study. Circulation 117, 743–753 (2008).

    Article  PubMed  Google Scholar 

  6. Conroy, R. M. et al. Estimation of ten-year risk of fatal cardiovascular disease in Europe: the SCORE project. Eur. Heart J. 24, 987–1003 (2003).

    Article  CAS  PubMed  Google Scholar 

  7. Graham, I. et al. Fourth Joint Task Force of the European Society of Cardiology and Other Societies on Cardiovascular Disease Prevention in Clinical Practice (constituted by representatives of nine societies and by invited experts). Eur. J. Cardiovasc. Prev. Rehabil. 14, E1–E40 (2007).

    Article  PubMed  Google Scholar 

  8. Yeboah, J. et al. Comparison of novel risk markers for improvement in cardiovascular risk assessment in intermediate-risk individuals. JAMA 308, 788–795 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  9. Goff, D. C. et al. 2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation 129, S49–S73 (2014).

    Article  PubMed  Google Scholar 

  10. Hira, R. S. et al. Frequency and practice-level variation in inappropriate aspirin use for the primary prevention of cardiovascular disease: insights from the National Cardiovascular Disease Registry’s Practice Innovation and Clinical Excellence registry. J. Am. Coll. Cardiol. 65, 111–121 (2015).

    Article  PubMed  Google Scholar 

  11. Cardiovascular Disease (10-Year Risk) (Framingham Heart Study, accessed 21 June 2017); https://www.framinghamheartstudy.org/risk-functions/cardiovascular-disease/10-year-risk.php

  12. Cooney, M. T. et al. How much does HDL cholesterol add to risk estimation? A report from the SCORE investigators. Eur. J. Cardiovasc. Prev. Rehabil. 16, 304–314 (2009).

    Article  PubMed  Google Scholar 

  13. Dudina, A. et al. Relationships between body mass index, cardiovascular mortality, and risk factors: a report from the SCORE investigators. Eur. J. Cardiovasc. Prev. Rehabil. 18, 731–742 (2011).

    Article  PubMed  Google Scholar 

  14. Wang, J. J. et al. Retinal vascular calibre and the risk of coronary heart disease-related death. Heart 92, 1583–1587 (2006).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Wong, T. Y. et al. Quantitative retinal venular caliber and risk of cardiovascular disease in older persons: the cardiovascular health study. Arch. Intern. Med. 166, 2388–2394 (2006).

    Article  PubMed  Google Scholar 

  16. Seidelmann, S. B. et al. Retinal vessel calibers in predicting long-term cardiovascular outcomes: the Atherosclerosis Risk in Communities Study. Circulation 134, 1328–1338 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  17. Wong, T. Y. et al. Retinal vascular caliber, cardiovascular risk factors, and inflammation: the multi-ethnic study of atherosclerosis (MESA). Invest. Ophthalmol. Vis. Sci. 47, 2341–2350 (2006).

    Article  PubMed  PubMed Central  Google Scholar 

  18. McGeechan, K. et al. Meta-analysis: retinal vessel caliber and risk for coronary heart disease. Ann. Intern. Med. 151, 404–413 (2009).

    Article  PubMed  PubMed Central  Google Scholar 

  19. McGeechan, K. et al. Prediction of incident stroke events based on retinal vessel caliber: a systematic review and individual-participant meta-analysis. Am. J. Epidemiol. 170, 1323–1332 (2009).

    Article  PubMed  PubMed Central  Google Scholar 

  20. Wong, T. Y. et al. Retinal arteriolar narrowing and risk of coronary heart disease in men and women. The Atherosclerosis Risk in Communities Study. JAMA 287, 1153–1159 (2002).

    PubMed  Google Scholar 

  21. Witt, N. et al. Abnormalities of retinal microvascular structure and risk of mortality from ischemic heart disease and stroke. Hypertension 47, 975–981 (2006).

    Article  CAS  PubMed  Google Scholar 

  22. Wong, T. Y. et al. Retinal microvascular abnormalities and 10-year cardiovascular mortality: a population-based case–control study. Ophthalmology 110, 933–940 (2003).

    Article  PubMed  Google Scholar 

  23. Cheung, C. Y.-L. et al. Retinal microvascular changes and risk of stroke: the Singapore Malay Eye Study. Stroke 44, 2402–2408 (2013).

    Article  PubMed  Google Scholar 

  24. Liew, G. et al. Fractal analysis of retinal microvasculature and coronary heart disease mortality. Eur. Heart J. 32, 422–429 (2011).

    Article  PubMed  Google Scholar 

  25. Kawasaki, R. et al. Fractal dimension of the retinal vasculature and risk of stroke: a nested case–control study. Neurology 76, 1766–1767 (2011).

    Article  CAS  PubMed  Google Scholar 

  26. Cheung, C. Y. et al. Retinal vascular fractal dimension and its relationship with cardiovascular and ocular risk factors. Am. J. Ophthalmol. 154, 663–674.e1 (2012).

    Article  PubMed  Google Scholar 

  27. Mookiah, M. R. K. et al. Computer-aided diagnosis of diabetic retinopathy: a review. Comput. Biol. Med. 43, 2136–2155 (2013).

    Article  PubMed  Google Scholar 

  28. Roychowdhury, S., Koozekanani, D. D. & Parhi, K. K. DREAM: diabetic retinopathy analysis using machine learning. IEEE J. Biomed. Health Inform. 18, 1717–1728 (2014).

    Article  PubMed  Google Scholar 

  29. LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).

    Article  CAS  PubMed  Google Scholar 

  30. Esteva, A. et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115–118 (2017).

    Article  CAS  PubMed  Google Scholar 

  31. Gulshan, V. et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316, 2402–2410 (2016).

    Article  PubMed  Google Scholar 

  32. Ting, D. S. W. et al. Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes. JAMA 318, 2211–2223 (2017).

    Article  PubMed  Google Scholar 

  33. Krause, J. et al. Grader variability and the importance of reference standards for evaluating machine learning models for diabetic retinopathy. Preprint at https://arxiv.org/abs/1710.01711 (2017).

  34. Simonyan, K., Vedaldi, A. & Zisserman, A. Deep inside convolutional networks: visualising image classification models and saliency maps. Preprint at https://arxiv.org/abs/1312.6034 (2013).

  35. Wong, T. Y., Klein, R., Klein, B. E. K., Meuer, S. M. & Hubbard, L. D. Retinal vessel diameters and their associations with age and blood pressure. Invest. Ophthalmol. Vis. Sci. 44, 4644–4650 (2003).

    Article  PubMed  Google Scholar 

  36. Angermueller, C., Pärnamaa, T., Parts, L. & Stegle, O. Deep learning for computational biology. Mol. Syst. Biol. 12, 878 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  37. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J. & Wojna, Z. Rethinking the inception architecture for computer vision. Preprint at https://arxiv.org/abs/1512.00567 (2015).

  38. Dean, J. et al. Large scale distributed deep networks. In Proc. 25th Conference on Advances in Neural Information Processing Systems (eds Pereira, F. et al.) 1223–1231 (Neural Information Processing Systems Foundation, 2012).

  39. Ioffe, S. & Szegedy, C. Batch normalization: accelerating deep network training by reducing internal covariate shift. Preprint at https://arxiv.org/abs/1502.03167 (2015).

  40. Russakovsky, O. et al. ImageNet Large Scale Visual Recognition Challenge. Int. J. Comput. Vis. 115, 211–252 (2015).

    Article  Google Scholar 

  41. Caruana, R., Lawrence, S. & Giles, L. Overfitting in neural nets: backpropagation, conjugate gradient, and early stopping. In Proc. 13th Conference on Advances in Neural Information Processing Systems (eds Leen, T. K. et al.) (MIT Press, Cambridge, MA, 2001).

  42. Krizhevsky, A., Sutskever, I. & Hinton, G. E. ImageNet classification with deep convolutional neural networks. In Proc. 25th Conference on Advances in Neural Information Processing Systems (eds Pereira, F. et al.) 1097–1105 (Neural Information Processing Systems, 2012).

  43. Xu, K. et al. Show, attend and tell: neural image caption generation with visual attention. Preprint at https://arxiv.org/abs/1502.03044 (2015).

  44. Cho, K., Courville, A. & Bengio, Y. Describing multimedia content using attention-based encoder–decoder networks. IEEE Trans. Multimed. 17, 1875–1886 (2015).

    Article  Google Scholar 

  45. Bahdanau, D., Cho, K. & Bengio, Y. Neural machine translation by jointly learning to align and translate. Preprint at https://arxiv.org/abs/1409.0473 (2014).

Download references

Acknowledgements

We thank C. Angermueller, A. Narayanaswamy, A. Rajkomar, A. Taly and P. Nelson from Google Research for their advice and assistance with reviewing the manuscript, and J. Cuadros for access to and clarifications on the EyePACS dataset and for thoughtful discussions. We used the UK Biobank Resource under application number 17643 for this work.

Author information

Authors and Affiliations

Authors

Contributions

R.P., A.V.V., Y.L., G.S.C., L.P. and D.R.W. designed the research; R.P., A.V.V., K.B., Y.L. and L.P. acquired data and/or executed the research; R.P., A.V.V., K.B., Y.L., M.V.M., L.P. and D.R.W. analysed and/or interpreted the data; R.P., A.V.V., K.B., Y.L., M.V.M., G.S.C., L.P. and D.R.W. prepared the manuscript.

Corresponding author

Correspondence to Lily Peng.

Ethics declarations

Competing interests

The authors are employees of Google and Verily Life Sciences.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Poplin, R., Varadarajan, A.V., Blumer, K. et al. Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning. Nat Biomed Eng 2, 158–164 (2018). https://doi.org/10.1038/s41551-018-0195-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41551-018-0195-0

Search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics