Machine Learning Better Predicts Suicide Risk After Emergency Department Visits

Matthew K. Nock, PhD

Investigators have developed a new machine learning technique to better predict high risk patients for suicide following an emergency department (ED) visit.

A team, led by Matthew K. Nock, PhD, Department of Psychology, Harvard University, identified a way to forecast suicide attempts within 1 and 6 months of presentation at an emergency department for psychiatric issues.

Research shows 50% of individuals who commit suicide make a health care visit within 1 month of death. There are a number of evidence-based interventions to reduce the risk of suicide, but to be expensive they must target high-risk patients.

The Study

In the prognostic study, the investigators assessed the 1-month and 6-month risk of suicide for 1818 patients presenting at the emergency department at Massachusetts General Hospital between February 4, 2015 and March 13, 2017. The median age of the study was 33 years.

The investigators sought main outcomes of suicide attempts 1 and 6 months following an ED visit, defined by combining data from electronic health records (EHR) with patient 1-month (n = 1102) and 6-month (n = 1220) through follow-up surveys.

The team used an ensemble machine learning technique to develop predictive models and a risk score for suicide.

In the month following the ED visit, 12.9% (n = 137) participants attempted suicide, while 22% (n = 268) attempted suicide within 6 months.

The investigators looked a clinician assessments and found these alone was a little better than chance at predicting suicide attempts, with externally validated area under the receiver operating characteristic curve of 0.67 for the 1-month model and 0.60 for the 6-month model.

Accuracy

However, prediction accuracy was slightly higher in the EHR models (1-month model: AUC, 0.71; 6 month model: AUC, 0.65) and was best using patient self-reports (1-month model: AUC, 0.76; 6-month model: AUC, 0.77). This was particularly true when patient-self-reports were combined with EHR data and/or clinician data (1-month model: AUC, 0.77; and 6 month model: AUC, 0.79).

The team also tested a model using the 20 patient self-report questions and an EHR-based risk score.

This model performed similarly well (1-month model: AUC, 0.77; 6 month model: AUC, 0.78).

However, the best 1-month model had a positive predicted value of 30.7% in patients classified as having the highest risk in the top 25% of the sample for suicide attempts, accounting for 64.8% of all 1-month attempts.

On the other hand, the best 6-month model had a 46% positive predicted value of patients of the highest risk class for suicide attempts, accounting for 50.2% of all 6-month attempts.

“The results of this prognostic study suggest that suicide risk assessments made using EHR-based and self-report–based risk scores may yield relatively accurate and clinically actionable predictions about the risk of suicide attempts by patients after presenting to an ED,” the authors wrote. “These results highlight the need for tests of the implementation of such risk assessment tools to target preventive interventions.”

The study, “Prediction of Suicide Attempts Using Clinician Assessment, Patient Self-report, and Electronic Health Records,” was published online in JAMA Network Open.