Steven Nissen, M.D., “scoffed at the analysis” of the Veterans Administration Diabetes Trial which showed statistically “significantly reduced rates of cardiovascular events with rosiglitazone,”1 before he even knew how it was done (See my post, Steven Nissen Calls the Kettle “Black”–VADT’s Avandia Analysis, January 22, 2014.).
So how were the Veterans Administration Diabetes Trial analyses of the safety of rosiglitazone (Avandia; GlaxoSmithKline) conducted, and what did they show?
There were three types of analyses2 (which I will explain as simply as I can):
- Case-control analysis
- Time-dependent covariate survival analysis
- Propensity matching analysis
Case-Control Analysis
In a case-control analysis, patients who have a particular event (cases) are matched to those who do not have that event (controls).
In this analysis of the Veterans Administration Diabetes Trial, the event categories were myocardial infarctions (heart attacks), strokes, death due to either of these causes, or major adverse cardiovascular events (any of the other events). Cases and controls were matched on prior history of myocardial infarctions and strokes, use of insulin at the beginning of the study, time to the event (up to the end of the study, for the controls), age, total cholesterol, and at least one other characteristic: duration of diabetes, obesity as measured by body mass index, systolic blood pressure, diastolic blood pressure, HDL (“good”) cholesterol, or HbA1c (an estimate of average blood glucose). The cases and controls were then compared on the number of visits for which rosiglitazone was prescribed and the average dose of rosiglitazone at each of those visits.2
For each of the four categories, the patients who had the “events” were prescribed rosiglitazone at fewer visits and at lower doses than those who did not have the “events.”2 While these results were not statistically significant (they could easily have occurred by chance), and while both the “events”2 and the lower doses and duration of rosiglitzone therapy might have been related due to episodes of hypoglycemia (low blood sugar), it is still telling that the results were opposite in direction to those predicted from Nissen and Wolski’s meta-analyses.3,4
Time-Dependent Covariate Analysis
In this analysis, the effect of rosiglitzone was examined with respect to the time until the first “event,” where the event categories were the same as for the case-control analysis, above.2 Time-dependent covariate analysis allows evaluation of the effects of variables whose values change with time,2 unlike the case-control analysis, above.
For both the 4 and the 8 mg doses of rosiglitazone, the risk ratio5 for first events was lower, regardless of whether or not the data was adjusted for patient characteristics at entry into the study (age, use of insulin, systolic blood pressure, and HDL), or characteristics both at entry into the study and throughout the study (age, use of insulin at entry, HDL, and HbA1c).2
For myocardial infarctions with the 4 mg dose, only the unadjusted risk ratio was statistically significant (in other words, unlikely to have occurred by chance alone), but with the 8 mg dose, the only the risk ratio that was not statistically significant was the one adjusted for characteristics both at entry and throughout the study. For strokes, cardiovascular deaths, and major adverse cardiovascular events, the risk ratios were statistically significantly lower with both doses of rosiglitazone, regardless of adjustment or non-adjustment.2 The higher the dose (0, 4, or 8 mg), the better rosiglitazone looked–not what one would expect from an unacceptably risky drug.
Interestingly, in each case, the “adjusted” and “unadjusted” results were almost identical.2
Just as importantly as their statistical significances, the sizes of each of these apparent differences were clinically highly significant.2
Note that statistical significance of the overall comparison (major adverse cardiovascular events) was justified comparison within its subclasses, “protecting” them from being falsely identified as “significant” as an artifact (artificial result) of multiple comparisons.6
Propensity Analysis
“Propensity analysis” is just a hard way of saying that those who were always prescribed rosiglitazone were compared with those who were never prescribed rosiglitazone.
Overall, there were generally small, but important, differences in for these groups’ characteristics, upon entering the study. Except for a nearly two-fold difference in smoking, these differences made the “events” less likely in those who were always treated with rosiglitazone than in those who never were.2
However, after matching them with a mathematical model (known as a “step-wise logistic regression”), the characteristics at entry into the study were almost identical between the two groups.2
Again, whether adjusted or not, the risk ratios for all four “events” were much lower in the group that had always been treated with rosiglitazone than in the group that never had been (The overall analysis, major adverse cardiovascular events, was statistically significant, justifying statistical testing of the subgroups, which were not significantly different6). Once again, the “adjusted” and “unadjusted” results were almost identical.2
Conclusions
“Based on the results from the VADT study, there is no evidence to suggest that use of rosiglitazone increases the risk for cardiovascular mortality or morbidity in patients with type-2 diabetes.”7
Actually, for cardiovascular results (heart attacks, strokes, related deaths, and the combination of the other results) that were uniformly clinically, and often statistically, significant in rosiglitazone’s favor,7 that was a major understatement–especially when, unlike Nissen and Wolski’s meta-analyses,3,4 the results were fairly robust, being demonstrated with very different approaches to the data.7 More to the point, “If anything, rosiglitazone had a protective effect.”1
Of perhaps the greatest importance, severe hypoglycemia (blood sugar low enough to require assistance from someone else) powerfully predicted cardiovascular events.1 Treating with higher doses, or with additional drugs, to lower the blood sugar will almost always increase the risk of hypoglycemia–which may be the “real” cause of any apparent increase in cardiovascular events from a diabetes drug. This may help to explain the general lack of benefit from “tight” blood glucose control seen in the Veterans Administration Diabetes Trial7 and multiple other studies. It might even be relevant to Nissen and Wolski’s own meta-analyses!3,4
No one claims that these unplanned analyses of the Veterans Administration Diabetes Trial are ideal. They were not even published with the other results.7 However, despite the limitations of these analyses, it is clear that the chief statistician for the Veterans Administration Diabetes Trial, Thomas E. Moritz, M.S., actually knows statistics,7 unlike heart doctor, Steven E. Nissen, M.D.8
“Statistical nonsense,” Dr. Nissen? It is time for you to stop calling kettles black.
© 2014 Myron Shank, M.D., Ph.D.9
1 Gever John. ADA: VA Diabetes Trial appears to vindicate rosiglitazone (Avandia) safety. MedPage Today June 8, 2008. http://www.medpagetoday.com/MeetingCoverage/ADA/9749
2 Moritz Thomas E. Impact of the use of rosiglitazone in VADT. http://www.fda.gov/downloads/AdvisoryCommittees/CommitteesMeetingMaterials/Drugs/EndocrinologicandMetabolicDrugsAdvisoryCommittee/UCM224743.pdf
5 See my post: What is the Risk of Avandia?, December 23, 2013.
6 See my post: Nissen, Wolski, and How Not to Do Meta-Analyses, December 19, 2013.
8 While Kathy Wolski, Dr. Nissen’s co-author for both his original5 and his updated6 rosiglitazone meta-analyses, is a statistician, not only is her opinion of the statistical analysis for the Veterans Administration Diabetes Trial not available, but the quality of her own work has been found seriously wanting7 (See also my posts: “Where’s the Beef?” for Avandia’s Risk?, January 13, 2014; Nissen and Wolski’s Avandia “Significance”, January 15, 2014; and “But wait! There’s more!” The RECORD on Avandia, January 17, 2014.). In particular, even though her own “updated” meta-analysis with Dr. Nissen6 was statistically non-significant,7 she was not deterred from claiming otherwise. While Dr. Nissen could (but will not) claim ignorance, she is a statistician and should be held to a higher standard. (For those who are wondering, yes, I have studied statistics, at both the collegiate and graduate levels.)
9 I served: in the speaker program for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, Chicago, Illinois, August 7, 1999; as a consultant for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, “Current Update and Discussion of the Glitazone Class of Oral Antidiabetics,” January 3, 2001; on Avandia Regional Advisory Panels in Cleveland, Ohio (February 4, 1999), Toledo, Ohio (September 15, 1999), San Juan, Puerto Rico (February 2000), and Beverly Hills, California (April 2000); as a preceptor for SmithKline Beechum, Lima, Ohio (June 23-25, 1999); and in a “Meet-the-Specialist,” SmithKline Beecham, Lima, Ohio, (2000). In 2000, SmithKline Beechum merged with GlaxoWellcome, to form GlaxoSmithKline. Before prescribing rosiglitazone (Avandia, Avandamet, Avandaryl; GlaxoSmithKline) became impractical (and then restricted), I prescribed both it and pioglitazone (ACTOS, ACTOSPlusMet, DuetAct; Takeda).
Dr. Shank 