Clinical Decision Support: Information overload can be a prescription for bad medicine

The push for clinical decision support technology in medicine is a logical consequence of our experiences as consumers and the need for intelligent support at the bedside. However, the reality is that despite its promise, clinical decision support technology has yet to demonstrate suffi ciant practical value in clinical practice.

Health information technology (HIT) has been offered as the panacea to all that ails modern healthcare. Promoters of HIT have predicted cost savings, reduced errors, improved outcomes, better patient satisfaction, increased standardization, satisfi ed providers, and world peace.^1—4 Alright, perhaps I exaggerated with the last one, but all of the others certainly have been touted as the benefi ts of HIT.

To be fair, the notion of clinical decision support (CDS) as a savior is incredibly “sticky.” Who could disagree when former Speaker of the House Newt Gingrich announced on June 6, 2005 on The Daily Show with Jon Stewart: “We think electronic health records could save— literally—between 40,000 and 100,000 lives per year.” It is easy for us to believe in these benefits because we experience the eff ect of information technology and decision support in all other areas of our lives. At home, my TiVo asks me if it is okay to overwrite another program when I request to record a new one, my online bank account lets me schedule automatic tasks, my Web browser suggests that I download its latest version, and my microwave suggests that I turn over my frozen chicken when defrosting it. We are used to decision support by IT—more, we expect it to make decisions easier, help scheduling processes, remind us of important events, and make workflow processes smoother. So, why not expect the same in medicine?

The push for CDS in medicine is a logical consequence of our experience in real life and the need for intelligent support at the bedside. However, the reality of clinical decision support is that it's a far cry from our experiences outside of medicine, and its fl aws and benefi ts have been only narrowly studied—25% of all studies on CDS have originated from only five US institutions.⁵ Clinical decision support suffers from four major challenges:

• Insufficient signal-to-noise ratio
• Insufficient resources for follow-up
• Lack of mature products
• Lack of financial incentives and legal protection

Signal-to-noise ratio

When a large East Coast academic medical center recently implemented a CPOE system with supposedly integrated CDS, one of the fi rst actions taken after going live was discontinuing drug—drug alerts. Even on the most stringent settings, the system would generate a plethora of alerts that would in 97.4% of cases be ignored by the user (who still had to “acknowledge” the alert with a click). In half of the remaining alerts, the alert would generate what would later be determined to be an error (users would discontinue Aspirin^® or Plavix^® on patients post-catheterization).⁶

Many of us can imagine scenarios in which an alert to a changing condition may be helpful. For example, would you want to be paged if your hospitalized patient’s WBC is 40,000? Before you say yes, what if you knew that yesterday's value was 45,000? What if you are an oncologist? The expected value of information from an alert depends on many factors: the patient’s diagnosis and age, the physician’s specialty, the location, and most importantly, prior knowledge. Indiscriminately fi ring alerts will not help providers; it will alienate them, and it will bury the truly important information among many other alerts. As you can tell from this scenario, in order to build a real alert, one must know more than the patient, the value, and whom to notify. One must consider prior knowledge and many other data in regards to the patient and institution. Is it more diffi cult than my Tivo checking if it is okay to overwrite my son’s recording of SpongeBob? Surely!

Insufficient resources for follow up Unless an alert is acted upon, it is worse than useless; it is a nuisance, a distraction, and a potential liability. Many CDS systems track alerts generated by the system. It would potentially be easy for a malpractice lawyer to imply negligence on the part of a physician who ignored an alert. However, when providers receive too many alerts, they are unable to respond to all of them due to time and productivity restraints. With one outpatient information system, providers received double-digit amounts of reminders on health maintenance issues and it was quickly discovered that although the alerts were designed to be valid for the patient, many of them were not acted upon.

Alerts can impact patient fl ow and resource utilization as well. In one emergency department study, a CDS system was abandoned because it strained resources by increasing patients’ length of stay and prompting physicians to order more tests.⁷ Having only 15 minutes with patients, physicians do not have time to respond to all possible clinical alerts. With that in mind, developers must from the beginning decide on how many and what type of alerts they want to serve.

Lack of mature products

The vast majority of institutions reporting on CDS have HIT systems that were developed in-house with CDS that is mostly hard-coded (not extractable) in their HIT systems. Products that are commercially available have not demonstrated much success in improving care over home-grown systems. Being able to understand the care processes in one institution and modeling CDS on these specifi c conditions may lead to a successful tool. Extracting this knowledge and making it applicable in many other institutions dilutes the essence of the CDS.

Lack of financial incentives and legal protection

Developing CDS is diffi cult and time consuming. Every alert must be analyzed; all possible conditions and responses must be modeled and then developed, tested, and implemented. This is a laborintensive process and until the product is implemented, nobody can predict if it will have the desired eff ect due to the inherent complexity of medicine. How many companies would be willing to gamble on such a risky investment? In addition, what are the consequences of developing CDS and making mistakes in the process? Therac-25, a radiation therapy device, injured several patients with lethal overdoses due to poorly designed error message alerts.⁸ What are the consequences to an institution that develops a CDS that inadvertently injures a patient?

CDS has a long way to go. Although it is not ready for general use yet, it still is a fascinating instrument that has enormous potential.

Christoph U. Lehmann, MD, is the director of clinical information technology at Johns Hopkins Children’s Medical and Surgical Center, past Education Chair of the AAP's COCIT Comittee, and is associate professor in Pediatrics & Health Sciences Informatics at Johns Hopkins University in Baltimore, MD. George Kim, MD, is a member of the executive committee and abstract chair for the AAP’s COCIT board.

REFERENCES

¹ www.ackdjournal.org/article/S1548-5595(07)00144-9/abstract

² www.jtrauma.com/pt/re/jtrauma/abstract.00005373-200802000-00049.htm

³ www.ncbi.nlm.nih.gov/pubmed/18085031

⁴ http://content.nejm.org/cgi/content/extract/348/25/2526

⁵ www.amia.org/inside/initiatives/cds

⁶ NN. Report on the implementation of a Drug-Drug alert. Personal communication, 2007.

⁷ www.ncbi.nlm.nih.gov/pubmed/17947627

⁸ http://courses.cs.vt.edu/~cs3604/lib/Therac_25/Therac_1.html