Accuracy of Smartphone-Based AI System for Diabetic Retinopathy

A new study is showing promise for the potentials of artificial intelligence (AI) in ophthalmology, after finding that a smartphone-based fundus camera was effective at diagnosing diabetic retinopathy in patients.

Results of the study, which included a total of 255 patients, revealed sensitivity and specificity of the offline AI system were greater than 85% in diagnosing referable diabetic retinopathy and diabetic retinopathy when compared with an ophthalmologist.

Investigators conducted a prospective, cross-sectional, population-based study to assess whether an offline automated analysis of retinal images in a smartphone could be a way to provide cost-effective and scalable method of screening for patients at risk of diabetic retinopathy. To assess the effectiveness of the AI system, investigators used the Remidio Non-Mydriatic Fundus on Phone to take images and the Medios AI — a proprietary offline automated analysis of retinal images on a smartphone to detect referable diabetic retinopathy — to analyze the retinal images and compared the sensitivity and specificity of the system to an ophthalmologist grading the same images.

Images used in the study were taken by a health care professional; with no previous experience using fundus cameras. Investigators enrolled 255 patients with diabetes mellitus, of which a cohort of 231 consented to diabetic retinopathy screening, and preliminary data including age, sex, duration since diabetes onset, and postprandial blood glucose level, were collected on all patients.

The 24 patients who declined to participate cited unwillingness to wait for screening and the blurring of vision that would sometimes occur after dilation. Images from one or both eyes of 18 patients were considered upgradable by the ophthalmologists and those images were excluded from analyses — a total of 213 patients were included in the final analysis. 



The mean age of patients included in the study was 53.1 years and more than half (110) of the patients were female. The mean postprandial blood glucose level was 207.8 mg/dL and the mean duration since diabetic onset was 5.5 years. 



After ophthalmologist grading, 187 patients were diagnosed ad having no diabetic retinopathy. Of these 187, a group of 172 (92%) patients were correctly diagnosed by the AI system while 15 were incorrectly diagnosed as having referable diabetic retinopathy.

A total of 15 (8%) patients were identified as having referable diabetic retinopathy via ophthalmologist grading and, investigators noted, the AI system correctly diagnosed all 15. A group of 12 individuals with cases of mild nonproliferation diabetic retinopathy were identified by the ophthalmologists — of these, the AI system diagnosed 8 (67%) as having referable diabetic retinopathy and 4 were diagnosed as not having diabetic retinopathy. 



Based on these results, investigators calculated the AI system’s sensitivity and specificity of diagnosing referable diabetic retinopathy as 100% (95% CI: 78.2% - 100%) and 88.4% (95% CI: 83.16% - 92.53%), respectively. Investigators added that the same values for any diabetic retinopathy were 85.2% (95% CI: 66.3% - 95.8%) and 92.0% (95% CI: 97.1% - 95.4%), respectively.

Additional analyses, using all images taken showed similar results. Sensitivity of the AI system was remained 100% (95% CI: 78.2% - 100.0%), but the specificity dropped down to 81.9% (95% CI: 75.9% - 87.0%) due to an increase in the number of non-diabetic retinopathy cases graded. 


In an invited commentary published in JAMA Ophthalmology, T.Y. Alvin Liu, MD, assistant professor of ophthalmology at Johns Hopkins Bloomberg School of Public Health, expressed both excitement and skepticism over the possibility of an affordable, yet effective screening system for diabetic retinopathy.

“This paradigm-shifting approach to DR screening could greatly benefit rural populations in both developing and developed countries where the access to care is limited,” Liu wrote. “One major drawback of this study is the wide CI estimates for sensitivity and specificity because of the small sample size, an issue that is currently addressed by an ongoing validation study with a much larger sample.”

This study, “Diagnostic Accuracy of Community-Based Diabetic Retinopathy Screening With an Offline Artificial Intelligence System on a Smartphone,” was published online in JAMA Ophthalmology.