DIGAMI 2 trial post hoc analysis: Lessons in overinterpretation

In their post hoc analysis of the DIGAMI 2 (Diabetes Mellitus Insu­lin-Glucose Infusion in Acute Myo­cardial Infarction 2) study, Mellbin and colleagues suggest that insulin therapy after myocardial infarction (MI) may be associated with increased clinical events (not mortality), whereas metformin thera­py may be associated with reduced events and sulfonylurea therapy with neutral ef­fects.¹ Several factors must be considered, however, before this somewhat dogmatic conclusion regarding insulin treatment can be accepted. This commentary focuses on the risks of overinterpretation of post hoc analyses, especially when they are not pre­specified analyses. It also provides a brief review of issues regarding the original DIGAMI study, shedding light on why the authors’ conclusions about insulin must be interpreted cautiously.² Finally, a brief summary of insulin and glycemic control in post-MI patients is provided to put the authors’ review in context.^3,4

Guidelines for conducting post hoc analyses

What are the generally acceptable guidelines for conducting post hoc anal­yses of a clinical trial? First, when the primary outcome of the trial is positive (either for efficacy or safety), then analy­ses examining differences in subgroups are often performed for age, gender, and medication use to determine whether sim­ilar outcomes are seen in all subgroups. Overlapping confidence intervals (CIs) im­ply no differences among subgroups. Even when “no difference” is demonstrated, far too often there is an overinterpretation of “differences” in point estimates. An even more serious flaw occurs when looking for differences in subgroups when no dif­ferences existed in the primary outcome analyses. This is precisely what Mellbin and associates have done. Finally, use of secondary end points in post hoc analyses, especially when they were negative in the original report, is especially prone to overinterpretation. In DIGAMI 2, the primary end point was all-cause mortality.² In the current report, there are no differences in all-cause mortality between insulin-treated and noninsulin-treated patients. All of the differences in the current report are related to the secondary end points, even though the authors of the original DIGAMI 2 study did not report any differences in these secondary end points.

Why does this approach raise such a serious concern? When a trial demon­strates overall negative results, as with DIGAMI 2, looking for differences among the variables is potentially treacherous and heavily dependent on asking a suf­ficient number of queries and conducting adequate “data mining.” Second, when a substantial number of patients in the clini­cal trial are effectively excluded from the study, especially if they were at high risk for an adverse outcome (in the current study, subjects already deceased), analyses in the remaining subjects is substantially compromised. Finally, statistical adjustment for multiple variables is markedly limited with a small number of events.

Too many analyses can lead to flawed conclusions

There are some interesting and instructional reports on the results of analyses using variables to pre­dict outcomes that were unlikely related to the outcome(s) under evaluation. One of these came from the ISIS-2 (International Study of Infarct Survival 2) Collaborative Group.^5,6 In their review of intra­venous streptokinase and aspirin studies, the investigators performed several subgroup analyses, includ­ing one that examined the subjects’ astrological signs. They found that patients born under Gemini or Libra had a slight adverse effect on mortality with aspirin treat­ment compared with those born under other Zodiac signs in whom “a strikingly beneficial effect” was observed. This example very suc­cinctly highlights the risk of per­forming too many analyses. In their paper, the ISIS-2 investigators state: “It is, of course, clear that the best estimate of the real size of the treatment effect in each astrologi­cal subgroup is given not by the re­sults in that subgroup alone but by the overall results in all subgroups combined.” In a “negative” trial like DIGAMI 2, sufficient num­bers of analyses are likely to result in some intervention (in this case, insulin) that may simply show “ad­verse” effects as a play of chance. Similar points about multiple non­prespecified analyses have been noted by Austin and colleagues, who reported that Ontarians born under Leo had a higher probabil­ity of gastrointestinal hemorrhage and those born under Sagittarius had a higher probability of humer­us fractures.⁷ The authors noted: “Our analyses illustrate how the testing of multiple, nonprespecified hypotheses increases the likelihood of detecting implausible associa­tions. Our findings have important implications for the analysis and interpretation of clinical studies.” These examples illustrate the importance of caution when conduct­ing post hoc analyses.

Concerns about DIGAMI 2

A brief examination of DIGAMI 1 and DIGAMI 2 is needed to put the post hoc analy­sis of DIGAMI 2 in context.^2,8-10 Several reports from DIGAMI 1 found a reduction in mortality in patients receiving insulin after an MI.^8-10 Whether this reduction was related to insulin use or to effects on glycemic control could not be deduced; thus, DIGAMI 2 was designed, at least in part, to answer the question of whether acute or long-term insulin therapy (and measures of glycemic control) would reduce mortality after an MI. The study used 3 treatment strategies: (1) acute insulin-glucose infusion followed by insulin-based long-term glucose control; (2) insulin-glucose infusion followed by standard glucose control; and (3) routine metabolic management according to local practice. Power calculations determined that 3000 patients (1250 in the first 2 groups and 700 in the third group) were necessary to answer the research question; however, less than half (n = 1253) were actually random­ized when the study was closed by the steering committee because of “slow patient recruitment.” A trial underpowered to show efficacy of its prespecified primary (mortal­ity) and secondary (clinical events) outcomes because of a failure to achieve recruitment targets is prob­ably remiss in reporting an adverse outcome in post hoc analyses.

Hyperglycemia and MI: Further considerations

In 2006, Pittas and colleagues conducted a careful review of in-patient insulin use and the effects on outcomes in several patient popula­tions.⁴ The authors found a trend to­ward possible benefits of insulin af­ter MI (relative risk, 0.89; 95% CI, 0.76-1.03). In a recently published scientific review by Deedwania and colleagues for the American Heart Association, the authors note that although hyperglycemia is clearly associated with adverse outcomes after an MI, the data supporting glucose-lowering interventions are still not robust (level C evidence).³ They also acknowledge that insu­lin is the quickest way to achieve glycemic control in these high-risk patients. In light of the paucity of data in these high-risk patients, the authors state “there is an urgent need for definitive large randomized trials to determine whether treatment strategies aimed at glu­cose control will improve patient outcomes and to define specific glu­cose treatment targets.” Although Mellbin and colleagues also suggest the need for future studies, investi­gators who read their report may be dissuaded from considering insulin interventions in such studies. This possibility is disconcerting because it could reduce the likelihood of some carefully designed studies to answer important questions regard­ing the relationships of glucose re­duction and insulin use in post-MI patients.

Conclusions

Mellbin and colleagues’ post hoc analysis of DIGAMI 2 found that insulin therapy is potentially hazardous in patients who have had an MI, but this conclusion was based on a seriously flawed analy­sis. Future clinical trials of glyce­mic control in patients with acute coronary syndromes should not exclude insulin as a possible inter­vention based on this study. Well-designed and adequately powered studies are needed to determine the effect and role of insulin in acute coronary syndrome patients.