Autonomous Artificial Intelligence Improved Adherence to Diabetic Eye Disease Testing

T. Y. Alvin Liu, MD

Credit: Johns Hopkins University

Autonomous artificial intelligence systems improved testing adherence for diabetic eye diseases (DED) across primary care clinics, according to new research.¹

The results, presented at the 83rd Scientific Sessions of the American Diabetes Association (ADA 2023), suggested autonomous artificial intelligence decreased disparities in Black patients, who historically experience disadvantages in medical care.

“We looked at the real-world deployment at Johns Hopkins Medicine and found that autonomous artificial intelligence diabetic eye disease testing at primary care sites was associated with improved adherence, even after controlling for potential confounding factors,” presenting investigator T.Y. Alvin Liu, MD, assistant professor of ophthalmology, Wilmer Eye Institute, the Johns Hopkins School of Medicine, said at ADA 2023. “We also found that the deployment did improve access and equity, especially for patients of populations that were traditionally disadvantaged.”

Artificial intelligence aims to train a computer to act or think like a human; machine learning is a subtype that uses numerous data to train systems and deep learning acts as a subtype of machine learning. Recent advances in medical imaging analysis and language recognition are driven by deep learning. The US Food and Drug Administration (FDA) has approved the use of 3 autonomous artificial intelligence systems: the IDx-DR approved in 2018², EyeArt approved in 2020³, and AEYE Health in 2022⁴.

These systems have been deployed to sites including academia and private settings, artificial intelligence companies, and insurance payers. In recent years, academic organizations, including the ADA, have said an FDA-approved artificial intelligence system capable of detecting more than mild diabetic retinopathy could be used as an alternative to traditional screening approaches.¹

The current analysis examined changes in adherence to annual diabetic disease testing in an integrated healthcare system in Johns Hopkins Community Physicians ≥40 community-based primary care clinics before and after the deployment of autonomous artificial intelligence.

Investigators defined annual diabetic eye disease testing as a completed evaluation by either a human ophthalmology provider or autonomous AI within a given calendar year. During the COVID-19 pandemic period, autonomous AI was introduced at multiple clinics. By 2021, some clinics used autonomous artificial intelligence sites while others did not use artificial intelligence (non-AI sites).

For the trial protocol, investigators compared the overall adherence in 2019 for a pre-AI period and in 2021 for an artificial intelligence period and then stratified by demographics, using chi-square or Fisher’s exact test. Then, the changes from 2019 to 2021 in odds of adherence within each subgroup were assessed for significant differences by site type, using logistic regression with a site type-by-time interaction term.

Upon analysis, the overall adherence rate increased from 46.1% to 54.5% at artificial intelligence sites (n = ~5000 patients) from 2019 - 2021. In the same period, the overall adherence rate increased from 40.4% to 40.3% at non-AI sites (n = ~12,000 patients). The analysis showed the increase in overall adherence rates at artificial intelligence sites was significantly greater than that at non-AI sites (P <.001).

At artificial intelligence sites, results indicate the odds ratio (OR) of adherence to diabetic eye disease testing in 2021 compared to 2019 was 1.54 (95% CI, 1.42 - 1.66; P <.001). On the other hand, at non-artificial intelligence sites, the OR of adherence to diabetic eye disease testing was 1.13 (95% CI, 1.07 - 1.19; P <.001).

The investigative team noted the OR of adherence at artificial intelligence sites was 36% higher (OR, 1.36) than at non-artificial intelligence, indicating a higher increase in diabetic eye disease testing from 2019 - 2021 at artificial intelligence sites. Patient subgroups, including race, insurance coverage, and area deprivation index (ADI), at sites that deployed autonomous artificial intelligence, showed improved health equity.

Investigators looked at race with the largest difference in adherence rate at baseline, finding a 16% difference between Asian-American (61%) and African American (45%) patients. After the deployment of artificial intelligence, 57% of African American patients adhered to testing, and the difference decreased to 4%. Similar improvements were seen between those with military and Medicare insurance, as well as those in the 1st quartile and 4th quartile of ADI.

The team indicated the potential of a ceiling effect, as those who identified as Asian-American, those with military insurance coverage, and those in the 1st quartile of ADI, showed minimal improvements after the deployment of autonomous artificial intelligence.

“There is a ceiling effect that is not really explained by deployment,” Liu said. “We don’t know exactly what that is, but we have some therapies that maybe there are issues with accepting autonomous artificial intelligence across patient groups.”

References

1. Liu TYA, Huang J, Lehmann H, Wolf RM, Channa R, Abramoff MD. Autonomous Artificial Intelligence (AI) Testing for Diabetic Eye Disease (DED) Closes Care Gap and Improves Health Equity on a Systems Level. Oral presentation at the 83rd Scientific Sessions of the American Diabetes Association. June 23 – 26, 2023.
2. Commissioner O of the. FDA permits marketing of artificial intelligence-based device to detect certain diabetes-related eye problems. U.S. Food and Drug Administration. April 11, 2018. Accessed June 26, 2023. https://www.fda.gov/news-events/press-announcements/fda-permits-marketing-artificial-intelligence-based-device-detect-certain-diabetes-related-eye.
3. Eyenuk announces FDA clearance for EyeArt autonomous AI system for diabetic retinopathy screening. Eyenuk, Inc. ~ Artificial Intelligence Eye Screening. August 5, 2020. Accessed June 26, 2023. https://www.eyenuk.com/us-en/articles/diabetic-retinopathy/eyenuk-announces-eyeart-fda-clearance/.
4. Health A. Aeye Health receives FDA clearance for AI-based autonomous screening for referable diabetic retinopathy. PR Newswire: press release distribution, targeting, monitoring and marketing. November 15, 2022. Accessed June 26, 2023. https://www.prnewswire.com/news-releases/aeye-health-receives-fda-clearance-for-ai-based-autonomous-screening-for-referable-diabetic-retinopathy-301678515