Biases in Nissen and Wolksi’s Avandia Analysis?

Remember that Nissen and Wolski said that they used the Peto method,¹ but, “The patron saint of meta-analysis, Richard Peto, who helped bring the approach back from what we used to dismiss as ‘unplanned pooling,’ stressed approaches that would avoid bias . . . .”²

This post considers whether Nissen and Wolski avoided bias in their meta-analysis¹ of rosiglitazone (Avandia; GlaxoSmithKline).

“Bias” is a serious charge.  Is it justified?

Let us begin by consulting Steven Nissen, himself.  Dr. Nissen has blasted “Flaws, Bias, Misinterpretation and Fraud in Randomized Clinical Trials.”³  Some of his dishonorable mentions included:

• Use of unblinded study designs
• Ascertainment bias
• Errors of omission (selective reporting of results)
• Type I (particularly multiplicity) and Type II error

Let’s see how many of those applied to Nissen and Wolski’s own meta-analysis.¹

To begin with, Nissen and Wolski were unblinded, meaning they knew what effects their choices of studies would have:  “. . . this was not a blinded effort and the results were obtained with full knowledge of the effect of various analytic decisions on the outcome by the analysts.”²  In fact, they included the DREAM (Diabetes REduction Assessment With ramipril And rosiglitazone Medication) study in their meta-analysis because it suggested an increase in acute myocardial infarctions (heart attacks).

. . . in his presentation at the July 2010 AC [advisory committee of the Food and Drug Administration] meeting, Nissen made it clear that it was DREAM, with its adverse trends on AMI [acute myocardial infarction] as well as other endpoints (especially CHF [congestive heart failure]), that provoked the meta-analysis.²

Interestingly, Nissen uses his own unblinded study¹ to contrast with the unblinded RECORD (Rosiglitazone Evaluated for Cardiac Outcomes and Regulation of glycaemia in Diabetes) study.³

• Unblinded study?  Check one.

Nissen and Wolski¹ selectively chose studies to include, from among those that met their inclusion criteria,⁴ with the full knowledge of the impact that these choices would have upon the outcome:  “Nonetheless, it must be recognized that there was potential bias in its development (or at least the analysis was done with knowledge of what would emerge) . . . .”²

• Ascertainment bias?   Check another.

I devoted a whole post to Nissen and Wolski’s¹ strange omissions, in Strange Omissions in Nissen’s Avandia Analysis, January 31, 2014.  Suffice it to say that

There were, in fact, studies available to Nissen and to the public, which met Nissen’s inclusion criteria, but which he failed to include in his meta-analysis.  Nissen’s meta-analysis was thus biased by its failure to have conducted a complete, thorough review of the medical literature for qualifying RCTs [randomized controlled trials].⁴

• Errors of omission?  Check another one.

I devoted another whole post to Nissen and Wolski’s results depending upon the errors of multiple comparisons (multiplicity), in More Concerns With Nissen’s Avandia Analysis, January 29, 2014.  In brief, the more comparisons beyond one, the greater the bias toward finding a difference that does not really exist.  Nissen and Wolski reported eight.  Apparently, Dr. Nissen did not pay much attention to his own graph of the effect of multiple comparisons.

• Errors due to multiplicity?  Check one more.

Interestingly, although Dr. Nissen makes much of pharmaceutical sponsorship as being sufficient to prove bias,⁵ his list did not mention bias from sponsorships.  However, he did claim that “Companies are directed to pay any honoraria, speaking or consulting fees directly to charity so that neither income nor tax deduction is received.”³  This claim is not what it appears to be.  As will be seen in an upcoming post, Dr. Nissen’s hypocrisy in this.  Apparently, he has never been supported by GlaxoSmithKline, but

“. . . Steven Nissen has acknowledged that his work is supported by many GlaxoSmithKline competitors including Pfizer, AstraZeneca, Sanofi-Aventis, Eli Lilly and the Japanese drug company called Takeda, which makes the diabetes drug Actos that competes directly with Avandia.”⁶

• Sponsorship bias?   Add another.

While Dr. Nissen acknowledged limitations with the analysis, “his concern was mainly about the true magnitude of the risk, not whether or not the risk was present or absent.”²  Dr. Nissen did not mention it in his list,³ but it is bias, nonetheless.

• Bias toward a particular outcome?   Add a very big one.

How important were these biases to the results?  Considering “the fragility of their findings,”⁷ even a small effect from bias would devastate Nissen and Wolski’s meta-analysis–without considering all of the other profound flaws in their study.

In commenting on the Nissen meta-analysis, Psaty and Furburg⁷ noted that the weaknesses of the study were substantial and that “a few events either way might have changed the findings for myocardial infarction or death from cardiovascular causes.  In this setting, the possibility that the findings were due to chance cannot be excluded.”¹

Six different types of bias in just one study, four of which Dr. Nissen, himself, puts on a par with fraud, flaws, and misinterpretations.³

Maybe Dr. Nissen thinks that what is good for randomized controlled trials is too good for a meta-analysis¹ that was merely based upon  randomized controlled trials.

Or, maybe he is just biased.

© 2014 Myron Shank, M.D., Ph.D.⁸

¹ Nissen Steven E., Wolski Kathy.  Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes.  The New England Journal of Medicine 2007; 356:2457-2471.

² Temple Robert.  Memorandum to Janet Woodcock:  Data on Rosiglitazone, August 8, 2010.  http://www.fda.gov/downloads/drugs/drugsafety/postmarketdrugsafetyinformationforpatientsandproviders/ucm226066.pdf

³ Steven Nissen.  Flaws, bias, misinterpretation and fraud
in randomized clinical trials.  http://spo.escardio.org/eslides/view.aspx?eevtid=48&fp=3132

⁴ Schachtman Nathan A.  Learning to embrace flawed evidence–the Avandia MDL’s Daubert opinion. Schachtman Law January 10th, 2011. http://schachtmanlaw.com/learning-to-embrace-flawed-evidence-the-avandia-mdls-daubert-opinion/

⁵ Sullivan Thomas.  Nissen relationships with industry:  CME, professional societies and red dresses.  Policy and Medicine March 18, 2010.  http://www.policymed.com/2010/03/nissen-relationships-with-industry-cme-professional-societies-and-red-dresses.html (The reader is strongly encouraged to read this revealing article about Steven Nissen’s character and credibility.)

⁶ Milloy Steven.  Junk science: diabetes drug scare or scam?  FoxNews.com June 10, 2007. http://www.foxnews.com/story/2007/06/10/junk-science-diabetes-drug-scare-or-scam/

⁷ Psaty Bruce M., Furberg Curt D. Rosiglitazone and cardiovascular risk.  New England Journal of Medicine 2007; 356:2522-2524.

⁸ I served:  in the speaker program for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, Chicago, Illinois, August 7, 1999; as a consultant for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, “Current Update and Discussion of the Glitazone Class of Oral Antidiabetics,” January 3, 2001; on Avandia Regional Advisory Panels in Cleveland, Ohio (February 4, 1999), Toledo, Ohio (September 15, 1999), San Juan, Puerto Rico (February 2000), and Beverly Hills, California (April 2000); as a preceptor for SmithKline Beechum, Lima, Ohio (June 23-25, 1999); and in a “Meet-the-Specialist,” SmithKline Beecham, Lima, Ohio, (2000). In 2000, SmithKline Beechum merged with GlaxoWellcome, to form GlaxoSmithKline.  Before it became impractical to continue prescribing rosiglitazone (Avandia, Avandamet, Avandaryl; GlaxoSmithKline), I prescribed both it and pioglitazone (ACTOS, ACTOSPlusMet, DuetAct; Takeda).

Strange Omissions in Nissen’s Avandia Analysis

In my post of January 27, 2014, Concerns at FDA:  Nissen’s Avandia Analysis, I discussed concerns expressed by Deputy Director Robert Temple, in a  memo to Janet Woodcock, Director of the Food and Drug Administration’s Center for Drug Evaluation and Research:  Nissen and Wolski’s² peculiar and unblinded¹ selective choices from those studies that qualified for their meta-analysis,³ their potential bias in doing the analysis,¹ and their (improper) inclusion of DREAM (Diabetes REduction Assessment With ramipril And rosiglitazone Medication) after using it to develop their hypothesis.⁴  Dr. Temple concluded that “there was potential bias in its development (or at least the analysis was done with knowledge of what would emerge).”  Then, in my January 29, 2014 post, More Concerns With Nissen’s Avandia Analysis, I discussed some of Nissen and Wolski’s² improper use of statistics related to their multiple comparisons between those who were treated with and those who were treated without rosiglitazone (Avandia; GlaxoSmithKline).

Nissen and Wolski did not consider the combined endpoint of acute myocardial infarctions plus cardiovascular deaths, attributing this to uncertainty about the possibility of counting the same event twice.²  However, for any reasonable assumption about the occurrence of both acute myocardial infarction and cardiovascular death in the same patient, the effect would be much less than the problem of multiple comparisons, which they ignored.

Speaking of multiple comparisons, Steven Nissen’s “Flaws, Bias, Misinterpretation and Fraud in Randomized Clinical Trials” showed that he was aware of the problem, but, although he used his own meta-analysis² as an example to impugn another paper, Dr. Nissen hypocritically omited mention that he and Kathy Wolski² were guilty of the very same multiple comparisons that he decried in others.⁵

“Stroke is not mentioned at all” by Nissen and Wolski, “a strange omission as almost all CV [cardiovascular] event trials include it.”¹  Nissen and Wolski might have lacked the data, but, if so, they “apparently, did not seek it.”¹  Ironically, Nissen and Wolski² claim to have used Peto’s method in their meta-analysis,² but “Peto, for example, has emphasized the need to find critical missing data (strokes, for example, or time of events) . . . .”¹  Then again, as the Food and Drug Administration’s own analyses have shown, “rosiglitazone treatment was in fact associated with fewer strokes.”¹

Since Nissen and Wolski did not believe that they had the data to combine acute myocardial infarctions and did not evaluate strokes, it is no surprise that they did not address major adverse cardiac events¹ (acute myocardial infarctions plus strokes plus cardiovascular deaths), either.

Of course, acute myocardial infarctions can cause cardiovascular deaths, although not all cardiovascular deaths are caused by acute myocardial infarctions (Some, for example, are caused by strokes, which were curiously not even considered by Nissen and Wolski.).  Unless just one of these outcomes was counted per person, the number of independent observations (in statistical jargon, “n”) will also be exaggerated, giving us too much confidence in the apparent differences between groups.  Nissen and Wolski mentioned this problem, with respect to the measure, acute-myocardial-infarctions-plus-cardiovascular-deaths,² but the same problem would also have affected the techniques of multiple stage statistical testing (See More Concerns With Nissen’s Avandia Analysis, January 29, 2014.), had actually used such techniques for their multiple comparisons.  Once again, however, for any reasonable assumptions, the potential magnitude of the error would have been far less than the errors introduced by not correcting for multiple comparisons.

The Food and Drug Administration’s own meta-analysis of fifty-two studies⁶ included all of the measures that Nissen and Wolski ought to have included.  In the meta-analysis of fifty-two studies, acute myocardial infarctions were “nominally statistically significant,”⁷ strokes trended “favorably for rosiglitazone,” neither cardiovascular nor all-cause mortality was even close to being “nominally statistically significant,”⁷ and the major adverse cardiovascular events category was not statistically significant.  Unlike Nissen and Wolski,² The Food and Drug Administration at least acknowledged the impact of multiple comparisons:  “Any sort of correction for multiplicity leaves the finding well short of nominal statistical significance.”^1,7

Nissen and Wolski’s² “strange omissions” would have been bad enough, but there was plenty more.

© 2014 Myron Shank, M.D., Ph.D.⁸

¹ Temple Robert.  Memorandum to Janet Woodcock:  Data on Rosiglitazone, August 8, 2010.  http://www.fda.gov/downloads/drugs/drugsafety/postmarketdrugsafetyinformationforpatientsandproviders/ucm226066.pdf

² Nissen Steven E., Wolski Kathy.  Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes.  The New England Journal of Medicine 2007; 356:2457-2471.

³ Schachtman Nathan A.  Learning to embrace flawed evidence–the Avandia MDL’s Daubert opinion.  Schachtman Law January 10th, 2011. http://schachtmanlaw.com/learning-to-embrace-flawed-evidence-the-avandia-mdls-daubert-opinion/.

⁴ To review, I observed that “It is never appropriate to test a hypothesis with the same data that was used to generate it.”    For the statistically literate, I pointed out that excluding the statistical “degrees of freedom” that were improperly included from DREAM would, by itself, eliminate even the appearance of statistical significance.  I also noted that this was inexcusable from Dr. Nissen’s co-author, Kathy Wolski, since she is a statistician.

⁵ Steven Nissen.  Flaws, bias, misinterpretation and fraud in randomized clinical trials.  http://spo.escardio.org/eslides/view.aspx?eevtid=48&fp=3132

⁶ In other words, twenty-four percent more than the Nissen and Wolski meta-analysis.²

⁷ The phrases, “nominally statistically significant” and “nominal statistical significance” may refer to a “p” value of less than 0.05 or to the fact that an apparently statistically significant “p” value does not take into consideration complicating factors, such as multiple comparisons.  In the source memo, it is not always clear from the context whether the former, the later, or both meanings were intended.

⁸  I served:  in the speaker program for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, Chicago, Illinois, August 7, 1999; as a consultant for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, “Current Update and Discussion of the Glitazone Class of Oral Antidiabetics,” January 3, 2001; on Avandia Regional Advisory Panels in Cleveland, Ohio (February 4, 1999), Toledo, Ohio (September 15, 1999), San Juan, Puerto Rico (February 2000), and Beverly Hills, California (April 2000); as a preceptor for SmithKline Beechum, Lima, Ohio (June 23-25, 1999); and in a “Meet-the-Specialist,” SmithKline Beecham, Lima, Ohio, (2000).  In 2000, SmithKline Beechum merged with GlaxoWellcome, to form GlaxoSmithKline.  Before it became impractical to continue prescribing rosiglitazone (Avandia, Avandamet, Avandaryl; GlaxoSmithKline), I prescribed both it and pioglitazone (ACTOS, ACTOSPlusMet, DuetAct; Takeda).

More Concerns With Nissen’s Avandia Analysis

The patron saint of meta-analysis, Richard Peto, who helped bring the approach back from what we used to dismiss as ‘unplanned pooling,’ stressed approaches that would avoid bias and error, including selection of endpoints before the trials are analyzed and a reasonably high level of statistical significance.¹

Nissen and Wolski said they used the Peto method, for their original meta-analysis of rosiglitazone (Avandia; GlaxoSmithKline).²  So, how did they measure up?

As noted in my January 27, 2014 post, Concerns at FDA:  Nissen’s Avandia Analysis, Deputy Director Robert Temple, in a  memo to Janet Woodcock, Director of the Food and Drug Administration’s Center for Drug Evaluation and Research, was critical of Nissen and Wolski’s² peculiar and unblinded¹ selective choices of studies (from among those that met their inclusion criteria).³  Dr. Temple also raised questions about the (improper) inclusion in their meta-analysis of the DREAM study, on which their hypothesis had been based.⁴  About the Nissen and Wolski meta-analysis, Dr. Temple concluded that “there was potential bias in its development (or at least the analysis was done with knowledge of what would emerge).”¹

Nissen and Wolski reported doing eight different comparisons in their original meta-analysis.²  If these eight comparisons had been independent of (unrelated to) each other, the probability that they would have found a difference by chance, where none existed, would really have been 0.34,⁵ instead of the 0.05 that they claimed.²   In other words, instead of the probability of erroneously concluding, by chance, that there were differences between treatment with and without rosiglitazone being one-in-twenty, as they implied, the probability  would be more than one-in-three.

Unfortunately, acute myocardial infarctions and cardiovascular deaths are linked; whatever factors affect one, affect the other. “If the comparisons are not independent, it really is impossible to compute the probability . . . .”⁶  The truth lies somewhere between “p”=0.05 and “p”=0.34–a useless result.

One common technique is to make the threshold for statistical significance much stricter, adjusting  by dividing the required “p” value for significance by the number of comparisons made–in this case, 0.05/8=0.00625.⁷  This is the Bonferroni adjustment⁸  The Bonferroni adjustment is approximate and only appropriate if the comparisons are independent of each other.⁹

Some other techniques only test the individual components if the over-all comparison is significant–in other words, if (and only if) we know that there is a difference, we are entitled to find out where it is.  This is known as a multiple-stage test.¹⁰

There are other techniques, each of which has its own uses and limitations.¹⁰

Nissen and Wolski used none of these.

Return for concerns about what the Nissen and Wolski meta-analysis² left out.

© 2014 Myron Shank, M.D., Ph.D.¹¹

¹ Temple Robert.  Memorandum to Janet Woodcock:  Data on Rosiglitazone, August 8, 2010.  http://www.fda.gov/downloads/drugs/drugsafety/postmarketdrugsafetyinformationforpatientsandproviders/ucm226066.pdf

² Nissen Steven E., Wolski Kathy.  Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes.  The New England Journal of Medicine 2007; 356:2457-2471.

³ Schachtman Nathan A.  Learning to embrace flawed evidence–the Avandia MDL’s Daubert opinion.  Schachtman Law January 10th, 2011. http://schachtmanlaw.com/learning-to-embrace-flawed-evidence-the-avandia-mdls-daubert-opinion/.

⁴ As I pointed out, “It is never appropriate to test a hypothesis with the same data that was used to generate it.”  I also noted that this was inexcusable behavior from a statistician, such as Dr. Nissen’s co-author, Kathy Wolski.  For the statistically literate, I pointed out that excluding DREAM’s statistical “degrees of freedom,” which were improperly included, would have been sufficient, by itself, to have eliminated any pretense of statistical significance for Nissen and Wolski’s meta-analysis.

⁵ 1.00-0.95⁸=0.34.

⁶ Anonymous. The multiple comparisons problem. GraphPad Statistics Guide http://www.graphpad.com/guides/prism/6/statistics/index.htm?stat_the_problem_of_multiple_compar.htm. Accessed January 26, 2014.

⁷ The exact calculation is 1.00-0.95^-8=0.0064 (Notice the negative sign on the exponent; this is the 8th-root of 0.95.).

⁸ Anonymous.  Glossary of statistical terms:  “Bonferroni adjustment,” The Institute for Statistics Education.   http://www.statistics.com/index.php?page=glossary&term_id=611. Accessed January 26, 2014.¹¹ Again, this is inexcusable for a statistician, such as Kathy Wolski.

⁹ Anonymous.  Bonferroni.  Simple Interactive Statistical Analysis  http://www.quantitativeskills.com/sisa/calculations/bonhlp.htm>. Accessed January 26, 2014.

¹⁰ Anonymous. The GLM procedure:  multiple comparisons.  SAS/STAT User’s Guide. http://v8doc.sas.com/sashtml/stat/chap30/sect35.htm.  Accessed January 26, 2014.

¹¹  I served:  in the speaker program for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, Chicago, Illinois, August 7, 1999; as a consultant for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, “Current Update and Discussion of the Glitazone Class of Oral Antidiabetics,” January 3, 2001; on Avandia Regional Advisory Panels in Cleveland, Ohio (February 4, 1999), Toledo, Ohio (September 15, 1999), San Juan, Puerto Rico (February 2000), and Beverly Hills, California (April 2000); as a preceptor for SmithKline Beechum, Lima, Ohio (June 23-25, 1999); and in a “Meet-the-Specialist,” SmithKline Beecham, Lima, Ohio, (2000).  In 2000, SmithKline Beechum merged with GlaxoWellcome, to form GlaxoSmithKline.  Before it became impractical to continue prescribing rosiglitazone (Avandia, Avandamet, Avandaryl; GlaxoSmithKline), I prescribed both it and pioglitazone (ACTOS, ACTOSPlusMet, DuetAct; Takeda).

Are Avandia Advantages in VADT “Statistical Nonsense” (Nissen)?

Steven Nissen, M.D., “scoffed at the analysis” of the Veterans Administration Diabetes Trial which showed statistically “significantly reduced rates of cardiovascular events with rosiglitazone,”¹ before he even knew how it was done (See my post, Steven Nissen Calls the Kettle “Black”–VADT’s Avandia Analysis, January 22, 2014.).

So how were the Veterans Administration Diabetes Trial analyses of the safety of rosiglitazone  (Avandia; GlaxoSmithKline) conducted, and what did they show?

There were three types of analyses² (which I will explain as simply as I can):

• Case-control analysis
• Time-dependent covariate survival analysis
• Propensity matching analysis

Case-Control Analysis

In a case-control analysis, patients who have a particular event (cases) are matched to those who do not have that event (controls).

In this analysis of the Veterans Administration Diabetes Trial, the event categories were myocardial infarctions (heart attacks), strokes, death due to either of these causes, or major adverse cardiovascular events (any of the other events).  Cases and controls were matched on prior history of myocardial infarctions and strokes, use of insulin at the beginning of the study, time to the event (up to the end of the study, for the controls), age, total cholesterol, and at least one other characteristic:  duration of diabetes, obesity as measured by body mass index, systolic blood pressure, diastolic blood pressure, HDL (“good”) cholesterol, or HbA1c (an estimate of average blood glucose).  The cases and controls were then compared on the number of visits for which rosiglitazone was prescribed and the average dose of rosiglitazone at each of those visits.²

For each of the four categories, the patients who had the “events” were prescribed rosiglitazone at fewer visits and at lower doses than those who did not have the “events.”²  While these results were not statistically significant (they could easily have occurred by chance), and while both the “events”² and the lower doses and duration of rosiglitzone therapy might have been related due to episodes of hypoglycemia (low blood sugar), it is still telling that the results were opposite in direction to those predicted from Nissen and Wolski’s meta-analyses.^3,4

Time-Dependent Covariate Analysis

In this analysis, the effect of rosiglitzone was examined with respect to the time until the first “event,” where the event categories were the same as for the case-control analysis, above.²  Time-dependent covariate analysis allows evaluation of the effects of variables whose values change with time,² unlike the case-control analysis, above.

For both the 4 and the 8 mg doses of rosiglitazone, the risk ratio⁵ for first events was lower, regardless of whether or not the data was adjusted for patient characteristics at entry into the study (age, use of insulin, systolic blood pressure, and HDL), or characteristics both at entry into the study and throughout the study (age, use of insulin at entry, HDL, and HbA1c).²

For myocardial infarctions with the 4 mg dose, only the unadjusted risk ratio was statistically significant (in other words, unlikely to have occurred by chance alone), but with the 8 mg dose, the only the risk ratio that was not statistically significant was the one adjusted for characteristics both at entry and throughout the study.  For strokes, cardiovascular deaths, and major adverse cardiovascular events, the risk ratios were statistically significantly lower with both doses of rosiglitazone, regardless of adjustment or non-adjustment.²  The higher the dose (0, 4, or 8 mg), the better rosiglitazone looked–not what one would expect from an unacceptably risky drug.

Interestingly, in each case, the “adjusted” and “unadjusted” results were almost identical.²

Just as importantly as their statistical significances, the sizes of each of these apparent differences were clinically highly significant.²

Note that statistical significance of the overall comparison (major adverse cardiovascular events) was justified comparison within its subclasses, “protecting” them from being falsely identified as “significant” as an artifact (artificial result) of multiple comparisons.⁶

Propensity Analysis

“Propensity analysis” is just a hard way of saying that those who were always prescribed rosiglitazone were compared with those who were never prescribed rosiglitazone.

Overall, there were generally small, but important, differences in for these groups’ characteristics, upon entering the study.  Except for a nearly two-fold difference in smoking, these differences made the “events” less likely in those who were always treated with rosiglitazone than in those who never were.²

However, after matching them with a mathematical model (known as a “step-wise logistic regression”), the characteristics at entry into the study were almost identical between the two groups.²

Again, whether adjusted or not, the risk ratios for all four “events” were much lower in the group that had always been treated with rosiglitazone than in the group that never had been (The overall analysis, major adverse cardiovascular events, was statistically significant, justifying statistical testing of the subgroups, which were not significantly different⁶).  Once again, the “adjusted” and “unadjusted” results were almost identical.²

Conclusions

“Based on the results from the VADT study, there is no evidence to suggest that use of rosiglitazone increases the risk for cardiovascular mortality or morbidity in patients with type-2 diabetes.”⁷

Actually, for cardiovascular results (heart attacks, strokes, related deaths, and the combination of the other results) that were uniformly clinically, and often statistically, significant in rosiglitazone’s favor,⁷ that was a major understatement–especially when, unlike Nissen and Wolski’s meta-analyses,^3,4 the results were fairly robust, being demonstrated with very different approaches to the data.⁷  More to the point, “If anything, rosiglitazone had a protective effect.”¹

Of perhaps the greatest importance, severe hypoglycemia (blood sugar low enough to require assistance from someone else) powerfully predicted cardiovascular events.¹  Treating with higher doses, or with additional drugs, to lower the blood sugar will almost always increase the risk of hypoglycemia–which may be the “real” cause of any apparent increase in cardiovascular events from a diabetes drug.  This may help to explain the general lack of benefit from “tight” blood glucose control seen in the Veterans Administration Diabetes Trial⁷ and multiple other studies.  It might even be relevant to Nissen and Wolski’s own meta-analyses!^3,4

No one claims that these unplanned analyses of the Veterans Administration Diabetes Trial are ideal.  They were not even published with the other results.⁷  However, despite the limitations of these analyses, it is clear that the chief statistician for the Veterans Administration Diabetes Trial, Thomas E. Moritz, M.S., actually knows statistics,⁷ unlike heart doctor, Steven E. Nissen, M.D.⁸

“Statistical nonsense,” Dr. Nissen?  It is time for you to stop calling kettles black.

© 2014 Myron Shank, M.D., Ph.D.⁹

¹ Gever John.  ADA:  VA Diabetes Trial appears to vindicate rosiglitazone (Avandia) safety.   MedPage Today June 8, 2008.   http://www.medpagetoday.com/MeetingCoverage/ADA/9749

² Moritz Thomas E.  Impact of the use of rosiglitazone in VADT.  http://www.fda.gov/downloads/AdvisoryCommittees/CommitteesMeetingMaterials/Drugs/EndocrinologicandMetabolicDrugsAdvisoryCommittee/UCM224743.pdf

³ Nissen Steven E., Wolski Kathy.  Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes.  The New England Journal of Medicine 2007; 356:2457-2471.

⁴ Nissen Steven E., Wolski Kathy.  Rosiglitazone revisited:  an updated meta-analysis of risk for myocardial infarction and cardiovascular mortality.  Archives of Internal Medicine 2010; 170:1191-1201.

⁵ See my post: What is the Risk of Avandia?, December 23, 2013.

⁶ See my post: Nissen, Wolski, and How Not to Do Meta-Analyses, December 19, 2013.

⁷ Duckworth William, Abraira Carlos, Moritz Thomas, Reda Domenic, Emanuele Nicholas, Reaven Peter D., Zieve Franklin J., Marks Jennifer, Davis Stephen N., Hayward Rodney, Warren Stuart R., Goldman Steven, McCarren Madeline, Vitek Mary Ellen, Henderson William G., Huang Grant D., for the VADT Investigators.  Glucose control and vascular complications in veterans with type 2 diabetes.  New England Journal of Medicine 2009; 360:129-139.

⁸ While Kathy Wolski, Dr. Nissen’s co-author for both his original⁵ and his updated⁶ rosiglitazone meta-analyses, is a statistician, not only is her opinion of the statistical analysis for the Veterans Administration Diabetes Trial not available, but the quality of her own work has been found seriously wanting⁷  (See also my posts:  “Where’s the Beef?” for Avandia’s Risk?, January 13, 2014; Nissen and Wolski’s Avandia “Significance”, January 15, 2014; and “But wait! There’s more!” The RECORD on Avandia, January 17, 2014.).  In particular, even though her own “updated” meta-analysis with Dr. Nissen⁶ was statistically non-significant,⁷ she was not deterred from claiming otherwise.  While Dr. Nissen could (but will not) claim ignorance, she is a statistician and should be held to a higher standard.  (For those who are wondering, yes, I have studied statistics, at both the collegiate and graduate levels.)

⁹ I served:  in the speaker program for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, Chicago, Illinois, August 7, 1999; as a consultant for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, “Current Update and Discussion of the Glitazone Class of Oral Antidiabetics,” January 3, 2001; on Avandia Regional Advisory Panels in Cleveland, Ohio (February 4, 1999), Toledo, Ohio (September 15, 1999), San Juan, Puerto Rico (February 2000), and Beverly Hills, California (April 2000); as a preceptor for SmithKline Beechum, Lima, Ohio (June 23-25, 1999); and in a “Meet-the-Specialist,” SmithKline Beecham, Lima, Ohio, (2000).  In 2000, SmithKline Beechum merged with GlaxoWellcome, to form GlaxoSmithKline.  Before prescribing rosiglitazone (Avandia, Avandamet, Avandaryl; GlaxoSmithKline) became impractical (and then restricted), I prescribed both it and pioglitazone (ACTOS, ACTOSPlusMet, DuetAct; Takeda).

Steven Nissen Calls the Kettle “Black” (VADT’s Avandia Analyses)

Nissen’s Prescience of the Veterans Administration Diabetes Trial

Without ever having read the (at that time) unpublished results of the Veterans Administration Diabetes Trial (VADT),  “Dr. Nissen scoffed at the analyses” of safety of rosiglitazone (Avandia, GlaxoSmithKline) before it had even been presented.¹

“The VADT trial provides no useful insights into the cardiovascular safety of rosiglitazone,” Dr. Nissen said in an email.

“Approximately 80% of patients in both the intensive and standard treatment groups received rosiglitazone [Avandia; GlaxoSmithKline].²  Therefore, a statistical analysis of the safety of this drug is not possible from VADT,” he said.  “Any assertion that VADT demonstrates the safety of rosiglitazone is statistical nonsense.”¹

Oops, Dr. Nissen!

Actually, it is quite possible, inspite of Dr. Nissen’s statistical pontifications.

The Veterans Administration Diabetes Trial was designed to evaluate the effects of intensive and conventional glucose control for diabetes,³ not to compare drug regimens.⁴  However, after Nissen and Wolski’s original meta-analysis,⁵ the investigators “agreed to conduct” the “specific analysis of events with rosiglitazone” with their data.¹

Dr. Nissen “scoffed at the analyses Moritz reported,” which showed statistically “significantly reduced rates of cardiovascular events with rosiglitazone.”  Simply comparing the intensive with the conventional glucose control arms would have revealed nothing about rosiglitazone, since most of the patients in each arm received that drug at some point, but that is not what Mr. Moritz did.⁴

In my next post, I will explain how the Veterans Administration Diabetes Trial analyses of rosiglitazone’s safety were conducted and what they showed.

© 2014 Myron Shank, M.D., Ph.D.⁶

¹ Gever John.  ADA:  VA Diabetes Trial appears to vindicate rosiglitazone (Avandia) safety.   MedPage Today June 8, 2008.   http://www.medpagetoday.com/MeetingCoverage/ADA/9749

² I do not know how Dr. Nissen came up with these numbers.  “Almost every VADT patient received rosiglitazone sometime during the study.”³ The percentages steadily trended downward from approximately 90% in both groups, at the beginning of the study, to about 25 and 30% in the conventionally- and intensively-treated groups, respectively, by the study’s end.³  Apparently, there no time when the percentage approximated 80% in both groups, and, during most of the study, it was less than that in both groups.³

³ Moritz Thomas E.  Impact of the use of rosiglitazone in VADT.  http://www.fda.gov/downloads/AdvisoryCommittees/CommitteesMeetingMaterials/Drugs/EndocrinologicandMetabolicDrugsAdvisoryCommittee/UCM224743.pdf

⁴ Duckworth William, Abraira Carlos, Moritz Thomas, Reda Domenic, Emanuele Nicholas, Reaven Peter D., Zieve Franklin J., Marks Jennifer, Davis Stephen N., Hayward Rodney, Warren Stuart R., Goldman Steven, McCarren Madeline, Vitek Mary Ellen, Henderson William G., Huang Grant D., for the VADT Investigators.  Glucose control and vascular complications in veterans with type 2 diabetes.  New England Journal of Medicine 2009; 360:129-139.

⁵ Nissen Steven E., Wolski Kathy.  Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes.  The New England Journal of Medicine 2007; 356:2457-2471.

⁶ I served:  in the speaker program for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, Chicago, Illinois, August 7, 1999; as a consultant for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, “Current Update and Discussion of the Glitazone Class of Oral Antidiabetics,” January 3, 2001; on Avandia Regional Advisory Panels in Cleveland, Ohio (February 4, 1999), Toledo, Ohio (September 15, 1999), San Juan, Puerto Rico (February 2000), and Beverly Hills, California (April 2000); as a preceptor for SmithKline Beechum, Lima, Ohio (June 23-25, 1999); and in a “Meet-the-Specialist,” SmithKline Beecham, Lima, Ohio, (2000).  In 2000, SmithKline Beechum merged with GlaxoWellcome, to form GlaxoSmithKline.  Before prescribing rosiglitazone (Avandia, Avandamet, Avandaryl; GlaxoSmithKline) became impractical (and then restricted), I prescribed both it and pioglitazone (ACTOS, ACTOSPlusMet, DuetAct; Takeda).

Dr. Nissen: “Just Say No” to Learning About Avandia

As Steven Nissen, M.D., sees it, the only reason that the Food and Drug Administration followed through with its long-announced promise for an independent “re-adjudication” (re-judging) of the RECORD¹ study was that “the leadership of the division of the FDA responsible for drug regulation, the Center for Drug Evaluation and Research (CDER), is seeking to avoid accountability for its role in the Avandia tragedy.”²

Dr. Nissen requested time to present his analyses about rosiglitazone (Avandia, Avandamet, Avandaryl; GlaxoSmithKline) to the Food and Drug Administration, but when he told them what he had to say, they concluded that it would merely duplicate other presentations.³

Uh, oh.  Dr. Nissen is not happy.  According to him, “the meeting seems systematically designed to absolve the drug of harm and the CDER leadership of any responsibility for ignoring the public health hazard of Avandia.  The current effort is intended to ‘whitewash’ the Avandia scandal and re-write history.”  He alleges that this is because “The leadership of CDER was intensely embarrassed by these revelations and furious with us for publicly challenging the safety of Avandia (and indirectly the competence and integrity of CDER).”²

Really?  And we are to believe that the same Dr. Nissen whose meta-analyses, “history” shows, have not only failed scientific and statistical scrutiny,⁴ but, even worse, have not been supported by real studies⁵ (let alone his own “updated” meta-analysis⁶) was not the least bit fearful of further embarrassment⁷ from the RECORD data?  Are we also to believe that the man who worked for Takeda, maker of the rival pioglitazone (ACTOS, ACTOSPlusMet, DuetAct), but not for GlaxoSmithKline, manufacturer of  rosiglitazone (Avandia, Avandamet, Avandaryl), was not above attempting to “blackwash” his opponents?  Are we not to even consider that this Dr. Nissen, who has been so strident in his attacks on anyone who does not support him, might be blackmailing those who are reluctant to have him study their drugs with his intravascular (within blood vessels) ultrasound procedure?⁸

Remember, Dr. Nissen, that ““to censor public debate about a critical public health issue is unacceptable and represents a grave threat to academic freedom.”⁹  You said that.  Those are your own words.  You did not believe that they meant allowing the TIDE study to continue–and possibly embarrass you.¹⁰  Clearly, you did not believe that they meant that the Food and Drug Administration should keep its word and allow the RECORD study to be independently re-adjudicated–which did embarrass you.⁵  Did you protest when the Chief of the Department of Medicine gave the endocrinologists in your own Cleveland Clinic strict orders not to say a word about rosiglitazone, with either an explicit or implicit threat that exercise of their “academic freedom” would cost them their jobs¹¹–or were you afraid of being embarrassed?

Dr. Nissen, we have heard you–over and over again.  Until you have nothing new to say, you should do something new and say nothing.

© 2014 Myron Shank, M.D., Ph.D.

¹ Rosiglitazone Evaluated for Cardiac Outcomes and Regulation of Glycaemia in Diabetes

² Nissen Steven E.  Steven Nissen:  the hidden agenda behind the FDA’s new Avandia hearings. Forbes Pharma & Healthcare, May 23, 2013.  http://www.forbes.com/sites/matthewherper/2013/05/23/steven-nissen-the-hidden-agenda-behind-the-fdas-avandia-hearings/

³ Herper Matthew. The FDA responds to Steve Nissen’s criticism of upcoming Avandia meeting. Forbes Pharma & Healthcare, May 23, 2013.  http://www.forbes.com/sites/matthewherper/2013/05/23/the-fda-responds-to-steve-nissens-criticism-of-upcoming-avandia-meeting/

⁴ See my posts:  Steven Nissen and The Not-So-Great Avandia Controversy, December 18, 2013 https://drshank.wordpress.com/2013/12/18/steven-nissen-and-the-not-so-great-avandia-controversy/; Nissen, Wolski, and How Not to Do Meta-Analyses, December 19, 2013 https://drshank.wordpress.com/2013/12/19/nissen-wolsky-and-how-not-to-do-a-meta-analysis-2/; What is the Risk of Avandia?, December 23, 2013 https://drshank.wordpress.com/2013/12/23/what-is-the-risk-of-avandia/; “Where’s the Beef?” for Avandia’s Risk?, January 13, 2014 https://drshank.wordpress.com/2014/01/13/wheres-the-beef-for-avandias-risk/; Nissen and Wolski’s Avandia “Significance,” January 15, 2014 https://drshank.wordpress.com/2014/01/15/nissen-and-wolskis-avandia-significance/.  (For those who are wondering, yes, I have studied statistics at both the collegiate and graduate levels.)

⁵ See my posts:  ADOPT the DREAM of a RECORD for Avandia, December 26, 2013 https://drshank.wordpress.com/2013/12/26/adopt-the-dream-of-a-record-for-avandia/ and “But wait! There’s more!” The RECORD on Avandia, January 17, 2014 https://drshank.wordpress.com/2014/01/17/but-wait-theres-more/

⁶ Nissen Steven E., Wolski Kathy.  Rosiglitazone revisited: an updated meta-analysis of risk for myocardial infarction and cardiovascular mortality.  Archives of Internal Medicine 2010; 170:1191-1201.

⁷ “Nevertheless, the results of RECORD do not substantiate the findings from the Nissen/Wolski meta-analysis on myocardial infarction and cardiovascular death.”  Furthermore, there was a “favorable trend for rosiglitazone on all-cause mortality.”  (Unger Ellis F.  FDA cardiologist’s perspective on RECORD:  joint meeting of the Endocrinologic and Metabolic Drugs Advisory Committee and the Drug Safety and Risk Management Advisory Committee, Avandia® (rosiglitazone) – July 13-14, 2010. http://www.fda.gov/downloads/advisorycommittees/committeesmeetingmaterials/drugs/endocrinologicandmetabolicdrugsadvisorycommittee/ucm218485.pdf)

⁸ The story behind Dr. Nissen’s claims to have all payments from drug companies go to charities will have to wait for another post.  In the meantime, I disclose that I served:  in the speaker program for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, Chicago, Illinois, August 7, 1999; as a consultant for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, “Current Update and Discussion of the Glitazone Class of Oral Antidiabetics,” January 3, 2001; on Avandia Regional Advisory Panels in Cleveland, Ohio (February 4, 1999), Toledo, Ohio (September 15, 1999), San Juan, Puerto Rico (February 2000), and Beverly Hills, California (April 2000); as a preceptor for SmithKline Beechum, Lima, Ohio (June 23-25, 1999); and in a “Meet-the-Specialist,” SmithKline Beecham, Lima, Ohio, (2000).  In 2000, SmithKline Beechum merged with GlaxoWellcome, to form GlaxoSmithKline.  Before it became impractical to continue prescribing rosiglitazone (Avandia, Avandamet, Avandaryl; GlaxoSmithKline), I prescribed both it and pioglitazone (ACTOS, ACTOSPlusMet, DuetAct; Takeda).

⁹ Husten Larry.  EHJ editors rebuffed GSK efforts to suppress Nissen editorial on rosiglitazone.  Cardiobrief April 30, 2010.  http://cardiobrief.org/2010/04/30/ehj-editors-rebuffed-gsk-efforts-to-suppress-nissen-editorial-on-rosiglitazone/.

¹⁰ See my post:  Avandia’s TIDE Is Stopped, December 24, 2013.  https://drshank.wordpress.com/2013/12/24/avandias-tide-goes-out/

¹¹ I know several endocrinologists from the Cleveland Clinic Foundation.  After Dr. Nissen’s original meta-analysis came out, every one of them refused to say so much as a word about rosiglitazone, explaining that their department chairman had emphatically prohibited them from doing so (One acknowledged that their chairman had ordered them to stop using the drug.).  It is obvious why their chairman would forbid criticism of a powerful member of the institution; it is equally obvious that there would be no reason to prevent the endocrinologists from speaking out (or from continuing to use rosiglitazone), if they had supported Dr. Nissen.  Not one endocrinologist I know (and I have represented one out of every nine in the United States) supported Dr. Nissen, and many (including myself) have been outspoken critics.

“But wait! There’s more!” The RECORD on Avandia

A Recap

In my post, “Where’s the Beef?” for Avandia’s Risk?, I had several occasions to point out the weaknesses of both of Nissen and Wolski’s meta-analyses,^1,2 the fact that the review accompanying their first meta-analysis¹ had to admit “the fragility of their findings,”³ the fact that their own “updated meta-analysis”² could not confirm their original meta-analysis,¹ and the fact that the Food and Drug Administration’s review,⁴ after not only raising questions about both the quality of their first meta-analysis¹ and its integrity, concluded, “When considering all data available at this time [2010], however, the overall evidence of increased myocardial ischemic event risk with rosiglitazone is weak.”   In Nissen and Wolski’s Avandia “Significance”, January 15, 2014, I pointed out that Hiatt, Kaul, and Smith obtained very different results, in their meta-analysis⁵ of exactly the same set of data as Nissen and Wolski’s original¹ meta-analysis.

As the saying goes, “But wait!  There’s more!”

The Interim Analysis of RECORD

Unlike the meta-analyses that preceded it, RECORD was  a randomized study of rosiglitazone for type 2 (non insulin-dependent) diabetes mellitus (specifically Avandia; GlaxoSmithKline), as compared to metformin (Glucophage; Bristol Myers Squibb) and the various sulfonylureas (members of a group of drugs that increase insulin secretion).

According to the Food and Drug Administration’s own criteria, the interim analysis of RECORD’s results showed that rosiglitazone was not associated with an unacceptable increase in the risk of major adverse cardiovascular events.⁶  Not only was the prespecified hypothesis (that the relative risk of of cardiovascular hospitalization or death for rosiglitazone was greater than or equal to 1.2, as compared to metformin, sulfonylureas, or their combination) not confirmed, but it actually appeared that rosiglitazone might have less risk than the other drugs.  In other words, while not statistically significant, most of the trends favored rosiglitazone, not Nissen and Wolski.

There were challenges to the original evaluation of RECORD.  Although they were often overstated, some of the challenges had merit.   On the other hand, there were also strengths in the study.  I have listed a few of them, below:

• Randomization was preserved in RECORD.
• Unlike Nissen and Wolski’s meta-analyses,^1,2 statistical heterogeneity (attempting to draw conclusions by combining studies that are extremely unlike each other) was not a problem in RECORD.
• Unlike the observational studies included in Nissen and Wolski’s meta-analyses,^1,2 unmeasured confounders (factors) were not a problem in RECORD.
• There were more major adverse cardiovascular events (myocardial infarctions or “heart attacks,” anginal chest pain, strokes, and cardiovascular deaths) available for evaluation in RECORD than in all of the studies in the Food and Drug Administration’s 2007 meta-analysis, combined, but there was no evidence of an increased risk from rosiglitazone for any of them, except for congestive heart failure (which is also increased with pioglitazone, just as with troglitazone, before it was taken off of the market).
• Analyses by subgroups generally gave results similar to those obtained with the main analyses and for analyses for myocardial infarctions (“heart attacks”).
• Analysis based upon the time period during which patients were actually on the specified treatments were generally consistent with those based upon the treatments that were originally planned for them, showing that there was no difference between the effects of actual treatments and those to which patients had been randomly (by chance) assigned.
• Even the upper limits (highest value) of the estimates for major adverse cardiovascular events met the Food and Drug Administration’s own standards for showing that rosiglitazone was not associated with an unacceptable increased in the risk.
• The overall death rates favored treatment with rosiglitazone over treatment without rosiglitazone, the opposite direction of what Nissen and Wolski^1,2 claim.

The Re-Analysis of RECORD

The Food and Drug Administration commissioned the Duke University’s Clinical Research Institute to independently re-adjudicate (re-judge) the data from RECORD.  From everything that I can tell, they did a masterful job of it.  “This well-conceived and comprehensive re-adjudication of the RECORD trial mortality experience demonstrated no evidence of a difference in all-cause mortality, cardiovascular plus unknown-cause mortality or non-cardiovascular mortality between the RSG [rosiglitazone] and MET/SU [metformin/sulfonylurea] treatment arms of RECORD.”⁷  In fact, for every one of the numerous sensitivity analyses’ multiple comparisons (each of which increased the likelihood of a false finding against rosiglitazone;⁸ see “Nissen, Wolski, and How Not to Do Meta-Analyses“), a lower death rate was found with rosiglitazone than for metformin/sulfonylurea.⁹  For fatal and non-fatal myocardial infarctions (“heart attacks”) and strokes, alone or in combinations, there was no evidence for any difference between rosiglitazone and metformin/sulfonylureas.⁹  In the end, there was no evidence of an increased risk from rosiglitazone for all-cause mortality, major adverse cardiovascular events, strokes, or myocardial infarctions.¹⁰

Commentary

At best, the evidence against rosiglitazone was always doubtful, no matter how much the Nissen-Wolski duet acted otherwise.  As will become increasingly evident, at worst, Steven Nissen, himself, was doubtful.

© 2014 Myron Shank, M.D., Ph.D.

¹ Nissen Steven E., Wolski Kathy.   Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes.  The New England Journal of Medicine 2007; 356:2457-2471.

² Nissen Steven E., Wolski Kathy.  Rosiglitazone revisited:  an updated meta-analysis of risk for myocardial infarction and cardiovascular mortality.  Archives of Internal Medicine 2010; 170:1191-1201.

³ Psaty Bruce M., Furberg Curt D.  Rosiglitazone and cardiovascular risk.  New England Journal of Medicine 2007; 356:2522-2524.

⁴ Parks Mary H.  (Untitled).  In:  FDA Briefing Document Advisory Committee Meeting for NDA 21071 Avandia (rosiglitazone maleate) tablet July 13 and 14, 2010.  http://www.fda.gov/downloads/AdvisoryCommittees/CommitteesMeetingMaterials/Drugs/EndocrinologicandMetabolicDrugsAdvisoryCommittee/UCM218493.pdf

⁵ Hiatt William R., Kaul Sanjay, Smith Robert J. The cardiovascular safety of diabetes drugs—insights from the rosiglitazone experience. New England Journal of Medicine 2013; 369:1285-1287.

⁶ Mahoney Karen Murry.  Advisory committee clinical briefing document: preliminary Endocrine Medical Officer review of the RECORD (Rosiglitazone Evaluated for Cardiac Outcomes and Regulation of Glycemia in Diabetes) trial, and update on cardiovascular safety information from large clinical trials of rosiglitazone New Drug Application 21071 Avandia® (rosiglitazone maleate).  In:  FDA Briefing Document Advisory Committee Meeting for NDA 21071 Avandia (rosiglitazone maleate) tablet July 13 and 14, 2010. http://www.fda.gov/downloads/AdvisoryCommittees/CommitteesMeetingMaterials/Drugs/EndocrinologicandMetabolicDrugsAdvisoryCommittee/UCM218493.pdf

⁷ Dunnmon Preston M.  DCRP consult to review and to comment on both all-cause and CV deaths from the Rosiglitazone Evaluated for Cardiovascular Outcomes and Regulation of glycaemia and Diabetes trial (RECORD, BRL-049653/231), according to the Phase-I re-adjudication of the trial’s mortality outcomes by the Duke Clinical Research Institute (DCRI). http://www.fda.gov/downloads/advisorycommittees/committeesmeetingmaterials/drugs/endocrinologicandmetabolicdrugsadvisorycommittee/ucm354859.pdf

⁸ Remember that performing these “subgroup analyses” within a category that has already been found not to show a statistically significant difference risks making it appear that there are differences when there are none. See “Nissen, Wolski, and How Not to Do Meta-Analyses. The fact that this did not happen, despite so many comparisons, is testimony that the overall results were not even close.  Although I do not have access to their raw statistical analysis, the Clinical Research Institute described this as a “sensitivity analysis,” strongly suggesting not only that they recognized the problem, but that they also interpreted negative results despite multiple comparisons as demonstrating the robustness of their overall negative conclusions.

⁹ Dunnmon Preston M.  Center for Drug Evaluation and Research Division of Cardiovascular and Renal Products consultation for NDA 021071 SDN 1652.  In:  FDA Briefing Document Advisory Committee Meeting for NDA 21071 Avandia (rosiglitazone maleate) tablet July 13 and 14, 2010. http://www.fda.gov/downloads/AdvisoryCommittees/CommitteesMeetingMaterials/Drugs/EndocrinologicandMetabolicDrugsAdvisoryCommittee/UCM218493.pdf

¹⁰ Andraca-Carrera Eugenio, Soukup Mat, Chakravarty Aloka.  Background information for advisory committee on rosiglitazone: statistical briefing material for the Endocrinologic and Metabolic Drugs Advisory Committee and the Drug Safety and Risk Management Advisory Committee Meeting, June 5-6, 2013.  http://www.fda.gov/downloads/advisorycommittees/committeesmeetingmaterials/drugs/endocrinologicandmetabolicdrugsadvisorycommittee/ucm354859.pdf

Nissen and Wolski’s Avandia “Significance”

The word “significance” has many meanings.  Those varied meanings actually help the purposes of this post.

On one hand, “significance” refers to “importance.”¹ On the basis of their profoundly disruptive impact,² Nissen and Wolski’s claims certainly were important and worthy of our attention in this series of posts.  In that sense, Nissen and Wolski’s meta-analyses^3,4 about rosiglitazone (Avandia, Avandamet, Avandaryl; GlaxoSmithKline) had “significance.”  But what about the other meanings of “significance?”

More important, for the current purposes, is the definition of “significance” as “meaning.”¹  What is the meaning of Nissen and Wolski^3,4?  The answer to that question requires yet another definition of “significance.”  I ask those readers who are either uncomfortable with, or uninterested in, statistics to briefly bear with me, as I lay the groundwork for yet more debunking of Nissen and Wolski to come.

In statistical jargon, “significance” refers to those outcomes of specialized mathematical tests that suggest that differences between groups are unlikely to have occurred by chance alone.  In other words, if differences between groups as large as those that were observed would seldom occur by chance alone, they are said to be “statistically significantly different,” or just “significant.”  “Significance” in the statistical sense is no guarantee of “significance” in the sense of being meaningful, but, on the other hand, it is best to reserve judgement that results are meaningful, if they could easily have been produced by chance processes.  The estimate of how frequently results at least as different as those observed could have been produced by chance alone is conventionally referred to as “p” (for the “probability” of achieving them by chance, alone).  The threshold number for the outcomes of those mathematical test should be adjusted, depending upon how you want to balance the costs of possibly mistakenly accepting differences as real or of possibly mistakenly dismissing them as too likely attributable to chance.  Rather than bother with actually thinking about such important matters, however, most medical literature arbitrarily uses a “p” value of 0.05, which is equivalent to one chance in twenty of occurring by chance; in other words, odds of 19 to 1⁵ that something other than chance was at work in producing the results.

However, as those of you who are familiar with statistics already know, the most important test of “significance” is not a small “p” value, but actually replicating the results in multiple independent experimental studies (See my post, ADOPT the DREAM of a RECORD for Avandia,” December 26, 2013). This is especially true when the value of “p” was estimated from a meta-analysis, which is an “artificial” or “pretend” study.⁶

This is actually a very important matter.  The reason for it may be found in something known as “Bayes Theorem.”  Again, in deference to those who lack a strong statistical background (and may even want to keep it that way!), I will try to keep this very simple:  Starting with an estimate of how likely an idea (“hypothesis”) is to be true, you obtain a new estimate which, combined with the old estimate, makes it more likely or less likely than before that the idea is true.  Rather than relying upon mere impressions, however, Bayes Theorem provides a formal way to obtain a revised estimate of how likely the idea is to be true.  The revised estimate depends upon (1) the old estimate, (2) the new estimate, (3) the strength of the old estimate, and (4) the strength of the new estimate.   (S.B. Leavitt provides what may be the best explanation–without going into the mathematics involved–that I have seen of using Bayes theorem to make medical decisions.⁷)

Even if one accepts on blind faith (as many people evidently have) Nissen and Wolski’s own original³ (and dubious) estimate of how unlikely it was that they would have obtained their results by chance alone, and even if one refuses to recognize (as many people evidently have) the numerous alternative explanations (biases) for their results, Nissen and Wolski still provided only a weak initial estimate for the likelihood of only a very modest effect.  Combining Nissen and Wolski’s highly questionable original estimate³ with the information from additional studies makes it very likely that differences as large as Nissen and Wolski claimed would frequently occur by mere chance–even ignoring the serious flaws in their meta-analysis, without which they could not have reached the conclusions that they did (See my post, “Nissen, Wolski, and How Not to Do Meta-Analyses,” December 19, 2013.).

Hiatt, Kaul, and Smith obtained very different results, in their meta-analysis⁸ of exactly the same set of data as Nissen and Wolski’s original meta-analysis.³  What is more, when Nissen and Wolski added new studies to their original meta-analysis to produce their “updated meta-analysis,”⁴ even without correcting the serious mistakes of their original meta-analysis, no matter how much they claimed otherwise, they still completely wiped out the appearance of a “statistically significant” result (Again, see my post, “Nissen, Wolski, and How Not to Do Meta-Analyses,” December 19, 2013.).

Others refuted Nissen and Wolski.  Nissen and Wolski even refuted themselves.  Their results lacked “statistical significance” and were not meaningful.  in both senses, Nissen and Wolski’s prattings were not significant.

Stay tuned for RECORD and more bad news for Nissen and Wolski on rosiglitazone.

© 2014 Myron Shank, M.D., Ph.D.

¹ Anonymous.  Significance.  Dictionary.com.  http://dictionary.reference.com/browse/significance

² Initially on the sales, but later on the availability of rosiglitazone (Avandia, Avandamet, Avandaryl; GlaxoSmithKline).

³ Nissen Steven E., Wolski Kathy.  Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes.  The New England Journal of Medicine 2007; 356:2457-2471.

⁴ Nissen Steven E., Wolski Kathy.  Rosiglitazone revisited: an updated meta-analysis of risk for myocardial infarction and cardiovascular mortality.  Archives of Internal Medicine 2010; 170:1191-1201.

⁵ In other words, (20-1) to 1.

⁶ The reader should not overlook the fact that Nissen and Wolski’s meta-analyses were not even performed credibly.  See my post, “Nissen, Wolski, and How Not to Do Meta-Analyses,” December 19, 2013.

⁷ Leavitt S.B.  The 6 worst words in evidence-based medicine.  Pain-Topics News/Research Updates, Saturday, December 14, 2013.  http://updates.pain-topics.org/2013/12/the-6-worst-words-in-evidence-based.html?showComment=1387733972654#c3773157618242887352

⁸ Hiatt William R., Kaul Sanjay, Smith Robert J.  The cardiovascular safety of diabetes drugs—insights from the rosiglitazone experience.  New England Journal of Medicine 2013; 369:1285-1287.

“Where’s the Beef?” for Avandia’s Risk?

Readers may remember the 1984 through 1985 Wendy’s hamburger television commercials with the famous line, “Where’s the beef?”   The idea was that some other hamburgers were all bun and no meat–all show and no substance.  The same could be said for the controversies surrounding rosiglitazone (Avandia, Avandamet, Avandaryl; GlaxoSmithKline).

The evidence suggesting that rosiglitazone increases the risks of myocardial infarctions (“heart attacks”) and cardiovascular events (“heart attacks,” anginal chest pain, strokes, and sudden death) was always extremely weak¹ (See also my previous posts, “Steven Nissen and The Not-So-Great Avandia Controversy,” December 18, 2013; “Nissen, Wolski, and How Not to Do Meta-Analyses,” December 19, 2013; “What is the Risk of Avandia?” December 23, 2013; and “Avandia’s TIDE Is Stopped,” December 24, 2013.).

A reviewer for the Food and Drug Administration was critical of Nissen and Wolski’s original meta-analysis²

• for including highly heterogeneous studies  (They mixed studies that were extremely different from each other.)
• for combining head-to-head trials of drugs with trials in which the drugs were merely added on when others failed (They mixed studies that had a strong design with those that had a weak design, and then treated them as the same.)
• for arbitrarily excluding the half of the trials that lacked cardiovascular deaths (primarily “heart attacks”) (They biased their results against rosiglitazone.)
• for falsely implying a comparison between rosiglitazone and insulin that did not exist
• for otherwise misrepresenting the designs of the studies, and
• for only using a single statistical technique to analyze the results (In other words, even with all of their errors, they may not have obtained the results that they did, if they had used different–and arguably more appropriate–statistical techniques.).³

Basing their analysis primarily on short-term (six-month) studies was of questionable validity, in the first place.  To illustrate:  if Nissen and Wolski had been honest enough to look at the six-month time point of PROactive, a study of the rival drug, pioglitazone (Actos, Actosplus Met, Actosplus Met XR, Duetact, Osini; Takeda), their results would have suggested an increased cardiovascular risk  that was not apparent at the end of the study.³  As we will see, the same pattern applies to rosiglitazone (Avandia, Avandamet, Avandaryl, GlaxoSmithKline).  Short-term results, such as these, are largely due to chance fluctuations in the occurrence of rare evens; with the passage of time averages them out.  Besides, which matters for drugs that are intended to be used for many years–the results at the end of a few months, or those after a few years?

Even an editorial that accompanied Nissen and Wolski’s highly controversial original meta-analysis had to admit “the fragility of their findings.”⁴

Their results were fragile, indeed.  Even Nissen and Wolski were unable to get the same results in their own later meta-analysis.⁵  In response to Nissen and Wolski’s claims,^2,5 the Food and Drug Administration (FDA) did its own meta-analysis.⁶  It suggested that rosiglitazone might increase myocardial ischemia (mostly anginal chest pain from the heart), but not myocardial infarctions (“heart attacks”), all-cause deaths, or the combination of myocardial infarctions, strokes, and cardiovascular deaths.⁶  In fact, the Food and Drug Administration observed that, “When the meta-analysis was performed by placebo- vs active-controlled trials, the increase in ischemic risk was observed in only the placebo-controlled trials.  This finding suggests that the risk of myocardial ischemia is similar between rosiglitazone and other oral anti-diabetic agents to which it was compared (metformin and sulfonylureas).”⁷  Once again, Nissen and Wolski’s conclusions depended upon weak evidence.

“When considering all data available at this time [2010], however, the overall evidence of increased myocardial ischemic event risk with rosiglitazone is weak.”¹

What was true then, as we will see, is even more true, now.  No matter how much Steven Nissen, M.D., wants us to focus on outside appearances, a closer look shows that the evidence for increased myocardial infarctions (“heart attacks”) and myocardial ischemia (anginal chest pain from the heart) with rosiglitazone (Avandia, Avandamet, Avandaryl, GlaxoSmithKline) is all bun and no beef.

Even worse, as we will also see, Dr. Nissen’s less than honorable behaviors may stem from less than honorable motives to viciously attack rosiglitazone (Avandia, Avandamet, Avandaryl, GlaxoSmithKline) and spare pioglitazone (Actos, Actosplus Met, Actosplus Met XR, Duetact, Osini; Takeda).

© 2014 Myron Shank, M.D., Ph.D.

¹ Parks Mary H. (Untitled).  In:  FDA Briefing Document Advisory Committee Meeting for NDA 21071 Avandia (rosiglitazone maleate) tablet July 13 and 14, 2010.  http://www.fda.gov/downloads/AdvisoryCommittees/CommitteesMeetingMaterials/Drugs/EndocrinologicandMetabolicDrugsAdvisoryCommittee/UCM218493.pdf

² Nissen Steven E., Wolski Kathy.  Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes.  The New England Journal of Medicine 2007; 356:2457-2471.

³ Mele Joy, Sahlroot Todd.  Statistical review and evaluation:  addendum to review completed 6/4/07.  In:  FDA Briefing Document Advisory Committee Meeting for NDA 21071 Avandia (rosiglitazone maleate) tablet July 13 and 14, 2010.  http://www.fda.gov/downloads/AdvisoryCommittees/CommitteesMeetingMaterials/Drugs/EndocrinologicandMetabolicDrugsAdvisoryCommittee/UCM218493.pdf

⁴ Psaty Bruce M., Furberg Curt D.  Rosiglitazone and cardiovascular risk.  New England Journal of Medicine 2007; 356:2522-2524.

⁵ Nissen Steven E., Wolski Kathy.  Rosiglitazone revisited: an updated meta-analysis of risk for myocardial infarction and cardiovascular mortality.  Archives of Internal Medicine 2010; 170:1191-1201.

⁶ Mahoney Karen Murry.  Preliminary Endocrine Medical Officer review of the RECORD (Rosiglitazone Evaluated for Cardiac Outcomes and Regulation of Glycemia in Diabetes) trial, and update on cardiovascular safety information from large clinical trials of rosiglitazone new drug application 21071 Avandia® (rosiglitazone maleate) 9 Jun 2010.  In:  FDA Briefing Document Advisory Committee Meeting for NDA 21071 Avandia (rosiglitazone maleate) tablet July 13 and 14, 2010.  http://www.fda.gov/downloads/AdvisoryCommittees/CommitteesMeetingMaterials/Drugs/EndocrinologicandMetabolicDrugsAdvisoryCommittee/UCM218493.pdf

⁷ Parks Mary H.  Inter-office background memo and draft questions for July 13 and 14, 2010 Advisory Committee Meeting for Avandia® (rosiglitazone).  In:  FDA Briefing Document Advisory Committee Meeting for NDA 21071 Avandia (rosiglitazone maleate) tablet July 13 and 14, 2010. http://www.fda.gov/downloads/AdvisoryCommittees/CommitteesMeetingMaterials/Drugs/EndocrinologicandMetabolicDrugsAdvisoryCommittee/UCM218493.pdf

ADOPT the DREAM of a RECORD for Avandia

On the basis of early analyses, the Food and Drug Administration (FDA) decided in 2007 that rosiglitazone (Avandia, Avandamet, Avandaryl; GlaxoSmithKline) could remain on the market with a boxed warning and a Medication Guide highlighting the possibility of an increased risk of myocardial infarction (“heart attack”).  The Food and Drug Administration also required the manufacturer, GlaxoSmithKline, to perform a cardiovascular outcomes study to prospectively evaluate the risk from rosiglitazone.¹  In response, GlaxoSmithKline initiated the TIDE (Thiazolidinedione Intervention with Vitamin D Evaluation) trial.  It was randomized, double-blinded, and placebo-controlled with a planned enrollment of 11,680 type 2 diabetic patients that was designed to compare the effects of adding rosiglitazone, pioglitazone (ACTOS, Takeda), or placebo to other anti-diabetic therapies.  The main comparison was to be the time until the first occurrence of either cardiovascular death, nonfatal myocardial infarction (“heart attack”), or nonfatal stroke (collectively known as “major adverse cardiovascular events”).  The vitamin D portion of the study was intended to evaluate the effects of long-term vitamin D supplementation on death and cancer.¹ As I have previously posted (“Avandia’s TIDE is Stopped,” December 24, 2013), TIDE was cancelled by the Food and Drug Administration (FDA).

In addition to the meta-analysis of forty-two short term studies mentioned in my December 19, 2013 post, “A Little Background on the Big Avandia Controversy,” in 2009, the Food and Drug Administration (FDA) also evaluated three large, long-term, controlled studies: ADOPT (A Diabetes Outcomes Progression Trial), DREAM (Diabetes REduction Assessment with ramipril and rosiglitazone Medication), and an analysis of the then ongoing RECORD (Rosiglitazone Evaluated for Cardiac Outcomes and Regulation of glycaemia in Diabetes) study.¹

The 4351 patients in ADOPT had recently been diagnosed with type 2 diabetes mellitus and had only been treated with diet and exercise.¹  They were assigned randomly (by chance) to therapy with a single drug:  either glyburide (Micronase, Pfizer), metformin (Glucophage, Bristol-Myers Squibb), or rosiglitazone (Avandia, GlaxoSmithKline).  Until the end of the study, neither  the patients nor the investigators knew which patient was receiving which drug (a double blind active-controlled study).²  The main outcome was the amount of time that it took for fasting plasma³ glucose levels to increase to 180 milligrams/deciliter.  Half of the patients were treated for less than, and half for more than, 4.0 years.  There were 31 all-cause deaths each among those treated with metformin or glyburide and 34 among those treated with rosiglitazone; these small differences were not statistically “significant”–in other words, they could easily have occurred by chance.²

The 5269 patients in DREAM did not meet criteria for diabetes, but were at high risk for developing it, because they had either abnormally high fasting plasma³ glucose levels (impaired fasting glucose) or an abnormal spike in blood glucose levels after a standard dose of glucose (impaired glucose tolerance).  They were assigned by chance to either rosiglitazone or an inactive lookalike, and, until the end of the study, neither they nor the investigators knew to which group they were assigned (a randomized, double-blind, placebo-controlled study).  The study looked at the development of either diabetes or death (from any cause) while on treatment.⁴ The combined outcome of diabetes and death was significantly lower in the group treated with rosiglitazone, so the authors were justified in attempting to identify where the difference occurred (See the discussion of “protected” statistical testing in my December 19, 2013 post, “Nissen, Wolski, and How Not to Do Meta-Analyses.”).  Even though there were no differences in the overall cardiovascular events or the combination of myocardial infarction (“heart attack”), stroke, and cardiovascular death, the authors reported a significantly higher rate of congestive heart failure with rosiglitazone.  While the authors could legitimately examine categories within the “significant” difference of their combined diabetes and death outcome, I do not believe that they could properly examine subcategories (including myocardial infarction, stroke, and cardiovascular death) within a category that they had found to be “non-significant.”  At any rate, they did the analyses, finding no evidence for differences in myocardial infarctions (“heart attacks”), stroke, cardiovascular death, new angina (chest pain from the heart), or revascularization (percutaneous transluminal angioplasty=PCTA or coronary artery bypass grafting=CABG).

The 4447 patients in RECORD had type 2 diabetes treated with either metformin or a sulfonylurea.  They were randomly assigned to the addition of either rosiglitazone or to the combination metformin plus a sulfonylurea.  The prespecified primary endpoint was hospitalization or death from cardiovascular causes, with a predefined non-inferiority risk ratio of 1.20.⁵  Unfortunately, the study was not blinded—in other words, both study subjects and investigators knew drug assignments during the study.  The reason for this was the expected need to use insulin, especially in the non-rosiglitazone groups, due to the duration of the study, and the problems with using placebo insulin.⁶   In an interim analysis performed while the study was ongoing, there was no difference in hospitalizations and death between rosiglitazone and the combination of metformin plus a sulfonylurea.  321 study subjects treated with rosiglitazone and 323 treated without rosiglitazone died or were hospitalized due to cardiovascular causes,⁵ for a “non-significant” hazard ratio of 321/2220:323/2227=0.99.  Since the 95% confidence interval did not include the prespecified risk ratio of 1.20¹ (a 20% increase in risk), these results did not support the prespecified risk level.⁵  As explained in the discussion of “protected” statistical testing in my December 19, 2013 post, “Nissen, Wolski, and How Not to Do Meta-Analyses,” since the overall result was statistically “non-significant,” the authors were not justified in performing subgroup analysis to look for the sources of a difference that did not exist.  Even though they did so anyway (without adjusting for multiple comparisons or “multiplicity”), only the increased risk for congestive heart failure that resulted in death or hospitalization was considered by the authors to be more common in those treated with rosiglitazone (and that was already well-known), but not either cardiovascular death, myocardial infarction, or stroke.⁵  In the words of the Food and Drug Administration, “The U.S. Food and Drug Administration (FDA) has determined that recent data for rosiglitazone-containing drugs, such as Avandia, Avandamet, Avandaryl, and generics, do not show an increased risk of heart attack compared to the standard type 2 diabetes medicines metformin and sulfonylurea.  As a result, we are requiring removal of the prescribing and dispensing restrictions for rosiglitazone medicines that were put in place in 2010.”⁷

Except for RECORD, studies were not designed to evaluate the cardiovascular risk of rosiglitazone (although, in DREAM, major adverse cardiovascular events were adjudicated by an endpoints committee).¹  In spite of the deficiencies of these three studies, combined, they represented 15067 patients with a minimum duration of treatment of approximately 3 years.  Impressively, even though flawed statistical techniques were biased toward falsely finding differences where none existed, there was no indication for any increased cardiovascular risk from rosiglitazone, except for the already known increase in congestive heart failure.  It shares  that risk with pioglitazone (Actos, Actosplus Met, Actosplus Met XR, Duetact, Osini; Takeda)–and with troglitazone (Rezulin, Warner-Lambert) before them.

Even though I am discussing the final results of these three studies, please do not lose sight of the fact that there was never valid evidence of any problem with rosiglitazone, in the first place.  Interestingly, I am friends with a number of endocrinologists, like myself, from Dr. Nissen’s own institution of the Cleveland Clinic, and, from the beginning of this controversy, every one of them held him in disdain, if not outright contempt.  Why?  I do not wish to over-estimate Dr. Nissen’s knowledge of statistics, but it is even conceivable that he knew better.  Was he motivated by the fact that Takeda had contracted for him to study pioglitazone, but that GlaxoSmithKline had not contracted for him to study rosiglitazone?  To those who would dismiss this suggestion out of hand, I would point out that Dr. Nissen’s ongoing behaviors in this controversy have been anything but professional, much less scientific.

© 2013 Myron Shank, M.D., Ph.D.

¹ Parks Mary H. Introduction memorandum. Readjudication of the Rosiglitazone Evaluated for Cardiovascular Outcomes and Regulation of Glycemia in Diabetes Trial (RECORD): Joint Meeting of the Endocrinologic and Metabolic Drugs Advisory Committee and the Drug Safety and Risk Management Advisory Committee June 5—6, 2013. http://www.fda.gov/downloads/advisorycommittees/committeesmeetingmaterials/drugs/endocrinologicandmetabolicdrugsadvisorycommittee/ucm354859.pdf

² Kahn Steven E., Haffner Steven M., Heise Mark A., Herman William H., Holman Rury R., Jones Nigel P., Kravitz Barbara G., Lachin John M., O’Neill M. Colleen, Zinman Bernard, Viberti Giancarlo, for the ADOPT Study Group. Glycemic durability of rosiglitazone, metformin, or glyburide monotherapy. New England Journal of Medicine 2006; 355:2427-2443.

³The liquid portion of blood, minus the cells.

⁴ DREAM (Diabetes REduction Assessment with ramipril and rosiglitazone Medication) Trial Investigators, Gerstein H.C., Yusuf S., Bosch J., Pogue J., Sheridan P., Dinccag N., Hanefeld M., Hoogwerf B., Laakso M., Mohan V., Shaw J., Zinman B., Holman R.R. Effect of rosiglitazone on the frequency of diabetes in patients with impaired glucose tolerance or impaired fasting glucose: a randomised controlled trial. Lancet 2006; 368:1096-105.

⁵ Home Philip D., Pocock Stuart J., Beck-Nielsen Henning, Gomis Ramon, Hanefeld Markolf, Jones Nigel P., Komajda Michel, McMurray John J.V., for the RECORD Study Team.  Rosiglitazone evaluated for cardiovascular outcomes in oral agent combination therapy for type 2 diabetes (RECORD): a multicentre, randomised, open-label trial. The Lancet 2009; 373:2125-2135.

⁶ Unger Ellis F. 21-071; suppl 35, 36, 37 Avandia (rosiglitazone). June 15, 2010.  In:  FDA Briefing Document Advisory Committee Meeting for NDA 21071 Avandia (rosiglitazone maleate) tablet July 13 and 14, 2010.  http://www.fda.gov/downloads/AdvisoryCommittees/CommitteesMeetingMaterials/Drugs/EndocrinologicandMetabolicDrugsAdvisoryCommittee/UCM218493.pdf (I do not believe that this was sufficient reason for a non-blinded study, however.)

⁷ US Food and Drug Administration.  FDA Drug Safety Communication:  FDA requires removal of some prescribing and dispensing restrictions for rosiglitazone-containing diabetes medicines. Drug Safety and Availability, November 25, 2013. http://www.fda.gov/Drugs/DrugSafety/ucm376389.htm.