Concerns about Pain Guidelines: When Strong Claims Follow from Weak Evidence

The development of clinical medical-practice guidelines is a difficult and arduous process. Experts in various disciplines volunteer many hours of their time for medical research reviews and analyses and meetings with fellow guidelines-panel members, and painstakingly writing and reviewing the final report. However, there are many concerns about the quality of medical research serving as a basis for those guidelines and, as some critics have proposed, “many wrong, or at least unreliable, therapeutic answers are being generated due to biased studies, representing small numbers of patients, and using inappropriate analyses” [Altman 2002]. How does this impact the quality of clinical practice guidelines in the pain management field?

Some problems Clinical practice guidelines are systematically developed statements intended to assist healthcare providers with decisions about appropriate care for specific patient circumstances [Tricoci, et al. 2009]. Guidelines are supposed to be just that, guides, not rules [Shaneyfelt and Centor 2009]. According to some recent commentary, there are simply too many guidelines, often on the same topics, and too many are of dubious quality, delivering questionable directives rather than assistive statements for everyday clinical practice. Consequently, guidelines are not always adhered to by healthcare providers even though, once they are published, guideline recommendations cannot be ignored [Shaneyfelt and Centor 2009; Sniderman and Furberg 2009].

Guidelines are expected to be based on evidence rather than opinion. However, it has been suggested that, “Most current articles called ‘guidelines’ are actually expert consensus reports” [Shaneyfelt and Centor 2009]. At that, the experts often have biases affecting their opinions, which are rarely made fully transparent to the reader. Gaps often exist in available evidence, so judgments frequently preserve the previously held positions of panel members; in effect, “what is to be decided is often already decided with the selection of the deciders” [Sniderman and Furberg 2009]. Furthermore, the process of formulating guidelines is essentially unregulated, and any group of individuals can designate itself as a guideline-development panel. Different panels often examine the same disease condition or topic of concern and reach significantly disparate conclusions [Shaneyfelt and Centor 2009].

Additionally, guidelines usually derive their authority from the sponsoring organization and the prestige of the journals in which they are published. Once issued, those organizations become promoters and staunch defenders of the guidelines, and panel members become stakeholders in the uncontested acceptance of their recommendations [Sniderman and Furberg 2009]. Briefly, here are a couple of recent clinical pain management guidelines demonstrating some typical reasons for concern.

Opioid-prescribing guidelines lacking strong evidence

Clinical practice guidelines from the American Pain Society and the American Academy of Pain Medicine (APS/AAPM) were published in February 2009 following a two-year review of 8,000 studies [Chou et al. 2009a]. The guidelines panel provided 25 recommendations on the safe and effective long-term use of opioids for carefully selected patients with chronic noncancer pain (CNCP). The recommendations were broad and far-ranging, and the guidelines panel was both exuberant and exemplary in its thoroughness and transparency. Besides the 17-page guideline document itself [Chou et al. 2009a], there are 285 pages of supplemental documentation explaining the details of study methodology and consulted resources, as well as describing quite extensive gaps and deficiencies in available evidence [APS 2009; Chou et al. 2009b, 2009c].

Most important, panel members also were intellectually honest in conceding that they were hindered by an overall deficiency of evidence and the poor quality of what was available. In fact, the panel did not rate any of its recommendations as supported by high-quality evidence and only four recommendations were viewed as supported by even moderate-quality evidence yet, all but three carry a “strong recommendation” designation. Therefore, the benefits in terms of improved patient outcomes and the validity for everyday practice of strong recommendations based on weak evidence must be questioned.

For example, the panel noted that several patient-screening instruments are recommended for predicting potential prescription opioid misuse or abuse; however, they concede that clinical evidence for the external validity of these tools and their benefits for everyday patient care are still lacking. Similarly, other researchers examining the needs of family physicians for guidance in this area observed, “No well-tested, easily administered screening tool exists for detecting drug-seeking behaviors in primary care patients” [Gianutsos and Safranek 2008]. The American Pain Society, itself, had earlier noted that when it comes to identifying patients exhibiting “drug seeking behavior” there “are no diagnostic behavioral criteria or accepted medical definitions” [Paice 2007].

At face value, most of the recommendations in the new APS/AAPM guidelines appear to make common sense for prudent medical practice when it comes to opioid therapy. However, the panel acknowledged that, at present, clinical decisions regarding the use of opioids for chronic noncancer pain must be based on weak evidence. A most important concern is that very little research supports whether or not the panel’s recommendations will favorably impact therapeutic outcomes. All of this seems to suggest that the promulgation of guidelines in this area might have been premature; perhaps, the findings of the APS/AAPM panel would have served the field better by establishing a direction for further investigations to fill critical research voids that were uncovered by their very thorough examination of the literature.

Other guidelines on this subject had been developed in the past. In an online publication, Washington and Fanciullo [2008] examine eight earlier documents; however, only half actually use the term “guidelines” in their titles; the others designate their documents as “management strategies,” “recommendations,” or “model policy,” which more appropriately connote that the contents are suggestions rather than rules. Yet, in all of those documents, the lack of randomized controlled trials for analysis was acknowledged as problematic, and there were strong discrepancies across guidelines in methods for evaluating the strength of the evidence presented and in topics of concern that were included or excluded.

ECG screening for methadone prescribing questionable In March 2009, the Annals of Internal Medicine published cardiac safety guidelines for physicians prescribing methadone for pain or as opioid-addiction therapy [Krantz et al. 2009]. The authors included some, but not all, members of a panel that had been convened by the US Center for Substance Abuse Treatment (CSAT) to examine available evidence regarding problems of methadone-associated arrhythmia and develop government-endorsed guidelines.

The most controversial recommendation is that all patients prescribed methadone should have pretreatment electrocardiography (ECG) and follow-up ECGs at 30 days and annually thereafter. Possible actions if the rate-corrected QT interval on the ECG exceeds acceptable limits include discussion with the patient of methadone’s risks and benefits, more frequent monitoring, dose reduction, or stopping methadone therapy entirely. Additionally, clinicians are instructed to learn about interactions between methadone and other drugs that can prolong the QTc interval or slow elimination of methadone.

While attention to the cardiac health of patients prescribed any strong opioid medications would be prudent medical practice, there are important concerns that raise questions regarding the validity of this guideline. For example, in a rebuttal editorial accompanying publication of the guidelines, Marc Gourevitch, MD, of New York University stresses that, overall, the recommendation for ECG screening presented in the guidelines “ventures well beyond the evidence presented.” He raises many cogent points of contention [Gourevitch 2009]:

 While it is evident that methadone can affect cardiac electrical conduction in certain circumstances, “how often patients treated with methadone develop arrhythmia is not clear.”

 The panel members presented no clinical trials or decision models to explain the relative benefits and harms of ECG screening.

 Rather than using their literature review to frame a process for acquiring missing data, the panel members went ahead in developing a clinical practice guideline, which was drafted and completed in only a two-day meeting.

 Although other analyses of this topic specifically did not recommended routine ECG screening [Cruciani 2008; Methadone Mortality 2007; Pearson and Woosley 2005], the panel members do not explain why they reached a contrary conclusion.

 Many medications can cause QT prolongation; however, there are no published trials documenting that arrhythmia and sudden death can be averted by baseline ECG screening.

 The panel members provide no evidence to justify their recommended ECG screening frequency, or whether this is more critical for persons with existing cardiac risk factors.

Gourevitch concludes that “…plausible arguments can be made for and against routine ECG screening in methadone treatment. But before guidelines are implemented, research methods, such as decision analysis, should be brought to bear to permit clear appreciation of the tradeoffs, benefits, and harms of alternative screening strategies.” A series of letters raising additional concerns about these guidelines were just recently published by the Annals in August 2009 [Byrne et al. 2009].

Questions about how the guidelines were published in the first place cast a cloud of doubt over their probity. The guidelines article first appeared in December 2008 prior to print at the Annals website under the title, “QTc Interval Screening in Methadone Treatment: the CSAT Consensus Guideline.” This implied official CSAT approval and, although this was not the case, some readers were understandably misled and erroneously started putting the guidelines into clinical practice. Following protests to the journal’s editor by concerned parties, the article was withdrawn; however, it soon reappeared again at the journal’s website without mention of CSAT in its title. In the final print-edition of the paper, there is a note from the editor clearly specifying, “This publication is not a federal guideline” and to this day an official government position on this subject has not been finalized and published. The many concerns surrounding this subject raise an important question: Why were these guidelines published?

Much more work ahead

Examples of possibly “misguided guidelines” in the pain management field are not unique in all of medicine. A recent examination of the various cardiovascular practice guidelines from the American College of Cardiology (ACC) and the American Heart Association (AHA) found that only 11% of more than 2,700 guideline recommendations were supported by high-quality evidence, and roughly half had very limited scientific backing [Tricoci et al. 2009]. Instead, most recommendations were based on expert opinion essentially, subjective viewpoints.

Tricoci, et al. [2009] observe that, through the years there have been considerable increases in the number of guideline recommendations in the cardiology field, but these have not been uniformly supported by increases in the quality of evidence. And, they point out that the field of cardiology has an extensive pool of large-scale clinical trials to rely upon, which many other medical specialties are lacking. An implication is that, if achieving quality guidelines in a field like cardiology is problematic, it could be even more troublesome in the pain management arena, which has a less extensive and weaker foundation of high-quality evidence.

While some authors propose that their guidelines are non-binding “Clinical practice guidelines/recommendations are intended to enhance patient care and do not supplant clinical judgment” [Krantz et al. 2009] the recommendations often do become accepted as standards of patient care that are turned into performance measures used to critique the quality of healthcare delivery [Shaneyfelt and Centor 2009]. As such, for example, they may be used to establish policies affecting insurance reimbursement for recommended therapies or procedures [Tricoci et al. 2009]. Furthermore, guidelines may be submitted in courts of law as evidence, and adherence or nonadherence to the implied standard of care may be alleged to portray either defensible or negligent performance, respectively, by a healthcare provider under scrutiny.

Much of what is officially published as practice guidelines, might better serve the pain management community if presented as “suggestions,” “strategies,” “advice,” or even “guidance.” Language can and does make a difference, and those alternative terms denote helpful but optional treatment approaches rather than medically and legally binding standards of care.

This was the recent approach of a panel in the United Kingdom that convened to produce evidence-based guidelines for the management of cancer-related breakthrough pain in adults [Davies et al. 2009]. They found insufficient evidence for developing guidelines regarding individual interventions and appropriately avoided the term “guidelines” in their published paper. The group did arrive at 12 recommendations (as suggestions) regarding generic strategies considered as useful, but they clearly cautioned that those were based on low-level evidence (eg, case series and expert opinion).

Along those lines, Shaneyfelt and Centor [2009] strongly advance their view that, “If all that can be produced are biased, minimally applicable consensus statements, perhaps guidelines should be avoided completely.” They further propose that, unless there are improvements in the guideline-development process, clinicians and policy makers should reject calls for adherence to presently available guidelines. Recognizing the lack of high-quality evidence in many medical disciplines, in February Congress budgeted more than $1 billion to fund research comparing the effectiveness of various treatments in head-to-head comparisons. This was part of a larger strategy to eventually streamline healthcare costs; however, it is unknown how much of this will benefit the pain management field and, considering the vast number of studies to be done and the time required, this is but a very worthwhile first step on a long and expensive road ahead.

Adapted from “Misguided Pain Guidelines,” which originally appeared in the Pain Treatment Topics “e-Briefing Newsletter,” available online here.

References: Altman DG. Poor-quality Medical Research: What Can Journals Do? JAMA. 2002; 287(21):2765-2767. Available online here.

APS (American Pain Society). Evidence Review: Guideline for the Use of Chronic Opioid Therapy inC Noncancer Pain. Glenview, IL: American Pain Society; 2009. Available online here.

Byrne A, et al. Concerns about Consensus Guidelines for QTc Interval Screening in Methadone Treatment. Ann Int Med. 2009;151(3):216-219. Available online here. Access checked 8/24/09.

Chou R, Ballantyne JC, Fanciullo GJ, et al. Research Gaps on Use of Opioids for Chronic Noncancer Pain: Findings from a Review of the Evidence for an American Pain Society and American Academy of Pain Medicine Clinical Practice Guideline. J Pain. 2009b;10(2):147-159. Available online here.

Chou R, Fanciullo GJ, Fine PG, et al. Clinical Guidelines for the Use of Chronic Opioid Therapy in Chronic Noncancer Pain. J Pain. 2009a;10(2):113-230. Available online here.

Chou R, Fanciullo GJ, Fine PG, et al. Opioids for Chronic Noncancer Pain: Prediction and Identification of Aberrant Drug-related Behaviors: A Review of the Evidence for an American Pain Society and American Academy of Pain Medicine Clinical Practice Guideline. J Pain. 2009c;10(2):131-146. Available online here.

Cruciani R. Methadone: To ECG or Not to ECG…That is Still the Question. J Pain Sym Manag. 2008;36(5):545-552. Available online here.

Gianutsos LP, Safranek S. Is There a Well-tested Tool to Detect Drug-seeking Behaviors in Chronic Pain Patients? J Fam Prac. 2008;57(9):609-610. Available online here.

Gourevitch MN. First Do No Harm ... Reduction? Ann Intern Med. 2009;150(6). Available online here.

Krantz MJ, Martin J, Stimmel B, Mehta D, Haigney MCP. QTc Interval Screening in Methadone Treatment: Clinical Guidelines. Ann Intern Med 2009;150(6):387-395. Available online here.

Methadone Mortality - A Reassessment. Rockville, MD: Substance Abuse and Mental Health Services Administration, Center for Substance Abuse Treatment; 2007(Jul). Available online here.

Paice JA. Letter from President of the American Pain Society to the Washington State Agency Medical Directors Group (AMDG). July 19, 2007. Available online here.

Pearson EC, Woosley RL. QT Prolongation and Torsades de Pointes among Methadone Users: Reports to the FDA Spontaneous Reporting System. Pharmacoepidemiol Drug Saf. 2005;14:747-53.

Shaneyfelt TM, Centor RM. Reassessment of Clinical Practice Guidelines. JAMA. 2009;301(8):868-869. Available online here.

Sniderman AD, Furberg CD. Why Guideline-making Requires Reform. JAMA. 2009;301(4):429-431. Available online here.

Tricoci P, Allen JM, Kramer JM, Califf RM, Smith Jr SC. Scientific Evidence Underlying the ACC/AHA Clinical Practice Guidelines. JAMA. 2009;301(8):831-841. Available online here.

Washington TA, Fanciullo GJ. Evidence-based Guidelines for Opioid Therapy. EmergingSolutionsInPain.com. 2008(Nov). Available online here.

Stewart B. Leavitt, MA, PhD, is the Founder and Executive Director of Pain Treatment Topics. He has more than 25 years of experience in healthcare education and medical communications. He also was the founding editor of Addiction Treatment Forum, in 1992, and worked on that project for 15 years. Leavitt was educated in biomedical communications at the University of Illinois College of Medicine, Chicago, IL, and then served as an Officer in the US Public Health Service. His advanced degrees focusing on health and medical education are from Northwestern University, Evanston, IL.