In a memo to Janet Woodcock, Director of the Food and Drug Administration’s Center for Drug Evaluation and Research, Deputy Director Robert Temple, pointed out that “it is MACE [major adverse cardiovascular events] that seems like the most informative endpoint and it is the one regularly used to assess favorable CV [cardiovascular] effects of drugs and drug risks (e.g., in the diabetes guidance),”1 rather than the individual components (myocardial infarction, strokes, and deaths due to the other two).1 As we will see, however, the individual components “were critical in the Nissan meta-analysis”1 of rosiglitazone (Avandia; GlaxoSmithKline).
As Dr. Temple pointed out, the Food and Drug Administration’s unfavorable 2010 review of the safety of rosiglitazone “did not really consider the weight of the various lines of evidence in much detail nor did it focus on the particular findings that were most critical”1 in light of Nissen and Wolski’s original meta-analysis.2
What were his concerns?
To begin with, why did Nissen and Wolski pool the large, long-term, ADOPT (A Diabetes Outcome Progression Trial) and DREAM (Diabetes REduction Assessment With ramipril And rosiglitazone Medication) studies with “40 relatively small studies, most of them 6 months or shorter”?1 As Dr. Temple noted, this choice had “important consequences to the outcome” of the meta-analysis, but was “not fully explained in the paper.”1
Dr. Nissen makes much of the fact that RECORD (Rosiglitazone Evaluated for Cardiovascular Outcomes and Regulation of Glycemia in Diabetes) was not blinded. However, he knew the results of ADOPT and DREAM, “so that the effect of combining these studies with the smaller ones was clear.”1 Nissen and Wolski picked and chose among studies which met their inclusion criteria, with full knowledge of what effects these choices would have on the results of their meta-analysis,2 and “it must be recognized that there was potential bias in its development (or at least the analysis was done with knowledge of what would emerge)” by doing so.1
Not everyone is so diplomatic.
There were, in fact, studies available to Nissen and to the public, which met Nissen’s inclusion criteria, but which he failed to include in his meta-analysis. Nissen’s meta-analysis was thus biased by its failure to have conducted a complete, thorough review of the medical literature for qualifying RCTs [randomized controlled trials].3
In fact, according to Dr. Temple, Dr. Nissen has attributed his interest in rosiglitazone’s impact on acute myocardial infarctions on the six extra myocardial infarctions among patients treated with rosiglitazone in DREAM.1 “In any event, speculation aside, this was not a blinded effort and the results were obtained with full knowledge of the effect of various analytic decisions on the outcome by the analysts, a very common situation with meta-analyses.”1
“As noted above, in his presentation at the July 2010 AC [advisory committee] meeting, Nissen made it clear that it was DREAM, with its adverse trends on AMI [acute myocardial infarction] as well as other endpoints (especially CHF [congestive heart failure]), that provoked the meta-analysis. In such a case one could ask whether DREAM should have been included in the meta-analysis.1
One could ask, but the question would be merely rhetorical. It is never appropriate to test a hypothesis with the same data that was used to generate it (For the statistically literate, I would point out that just excluding DREAM’s statistical “degrees of freedom,” which were improperly included in Nissen and Wolski’s meta-analysis, would have been sufficient, by itself, to eliminate any pretense of statistical significance.). Maybe cardiologist Steven Nissen, M.D., can be excused for this breach of fundamental statistical principles, but his co-author, statistician Kathy Wolski, cannot be.4
As Dr. Temple observes, “That is, of course, not what we expect in designing and evaluating RCTs [randomized controlled trials], where we stress the need to make all analytic plans before results are in hand to avoid bias.”1 He continues: “I recognize the irony of this: the not very precise, not well-specified findings in the [Nissen and Wolski] meta-analysis are held to a lower standard than the later trials that seek to examine the concerns stimulated by the [Nissen and Wolski] meta-analysis.”1
Nissen and Wolski’s meta-analysis was slip-shod, unblinded, biased, and inconsistent. To defend its dubious conclusions, Dr. Nissen and his allies have had to resort to double standards.
There is more to come.
© 2014 Myron Shank, M.D., Ph.D.5
1 Temple Robert. Memorandum to Janet Woodcock: Data on Rosiglitazone, August 8, 2010. http://www.fda.gov/downloads/drugs/drugsafety/postmarketdrugsafetyinformationforpatientsandproviders/ucm226066.pdf
3 Schachtman Nathan A. Learning to embrace flawed evidence–the Avandia MDL’s Daubert opinion. Schachtman Law January 10th, 2011. http://schachtmanlaw.com/learning-to-embrace-flawed-evidence-the-avandia-mdls-daubert-opinion/.
4Others have found the quality of her work seriously wanting, as well (Duckworth William, Abraira Carlos, Moritz Thomas, Reda Domenic, Emanuele Nicholas, Reaven Peter D., Zieve Franklin J., Marks Jennifer, Davis Stephen N., Hayward Rodney, Warren Stuart R., Goldman Steven, McCarren Madeline, Vitek Mary Ellen, Henderson William G., Huang Grant D., for the VADT Investigators. Glucose control and vascular complications in veterans with type 2 diabetes. New England Journal of Medicine 2009; 360:129-139. See also my posts: Steven Nissen and The Not-So-Great Avandia Controversy, December 18, 2013; Nissen, Wolski, and How Not to Do Meta-Analyses, December 19, 2013; “Where’s the Beef?” for Avandia’s Risk?, January 13, 2014; Nissen and Wolski’s Avandia “Significance”, January 15, 2014; and “But wait! There’s more!” The RECORD on Avandia, January 17, 2014.). In particular, even though her own “updated” meta-analysis with Dr. Nissen was statistically non-significant, she was not deterred from claiming otherwise (Nissen Steven E., Wolski Kathy. Rosiglitazone revisited: an updated meta-analysis of risk for myocardial infarction and cardiovascular mortality. Archives of Internal Medicine 2010; 170:1191-1201.).
5 I served: in the speaker program for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, Chicago, Illinois, August 7, 1999; as a consultant for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, “Current Update and Discussion of the Glitazone Class of Oral Antidiabetics,” January 3, 2001; on Avandia Regional Advisory Panels in Cleveland, Ohio (February 4, 1999), Toledo, Ohio (September 15, 1999), San Juan, Puerto Rico (February 2000), and Beverly Hills, California (April 2000); as a preceptor for SmithKline Beechum, Lima, Ohio (June 23-25, 1999); and in a “Meet-the-Specialist,” SmithKline Beecham, Lima, Ohio, (2000). In 2000, SmithKline Beechum merged with GlaxoWellcome, to form GlaxoSmithKline. Before it became impractical to continue prescribing rosiglitazone (Avandia, Avandamet, Avandaryl; GlaxoSmithKline), I prescribed both it and pioglitazone (ACTOS, ACTOSPlusMet, DuetAct; Takeda).