Biases in Nissen and Wolksi’s Avandia Analysis?

Remember that Nissen and Wolski said that they used the Peto method,1 but, “The patron saint of meta-analysis, Richard Peto, who helped bring the approach back from what we used to dismiss as ‘unplanned pooling,’ stressed approaches that would avoid bias . . . .”2

This post considers whether Nissen and Wolski avoided bias in their meta-analysis1 of rosiglitazone (Avandia; GlaxoSmithKline).

“Bias” is a serious charge.  Is it justified?

Let us begin by consulting Steven Nissen, himself.  Dr. Nissen has blasted “Flaws, Bias, Misinterpretation and Fraud in Randomized Clinical Trials.”3  Some of his dishonorable mentions included:

  • Use of unblinded study designs
  • Ascertainment bias
  • Errors of omission (selective reporting of results)
  • Type I (particularly multiplicity) and Type II error

Let’s see how many of those applied to Nissen and Wolski’s own meta-analysis.1

To begin with, Nissen and Wolski were unblinded, meaning they knew what effects their choices of studies would have:  “. . . this was not a blinded effort and the results were obtained with full knowledge of the effect of various analytic decisions on the outcome by the analysts.”2  In fact, they included the DREAM (Diabetes REduction Assessment With ramipril And rosiglitazone Medication) study in their meta-analysis because it suggested an increase in acute myocardial infarctions (heart attacks).

. . . in his presentation at the July 2010 AC [advisory committee of the Food and Drug Administration] meeting, Nissen made it clear that it was DREAM, with its adverse trends on AMI [acute myocardial infarction] as well as other endpoints (especially CHF [congestive heart failure]), that provoked the meta-analysis.2

Interestingly, Nissen uses his own unblinded study1 to contrast with the unblinded RECORD (Rosiglitazone Evaluated for Cardiac Outcomes and Regulation of glycaemia in Diabetes) study.3

  • Unblinded study?  Check one.

Nissen and Wolski1 selectively chose studies to include, from among those that met their inclusion criteria,4 with the full knowledge of the impact that these choices would have upon the outcome:  “Nonetheless, it must be recognized that there was potential bias in its development (or at least the analysis was done with knowledge of what would emerge) . . . .”2

  • Ascertainment bias?   Check another.

I devoted a whole post to Nissen and Wolski’s1 strange omissions, in Strange Omissions in Nissen’s Avandia Analysis, January 31, 2014.  Suffice it to say that

There were, in fact, studies available to Nissen and to the public, which met Nissen’s inclusion criteria, but which he failed to include in his meta-analysis.  Nissen’s meta-analysis was thus biased by its failure to have conducted a complete, thorough review of the medical literature for qualifying RCTs [randomized controlled trials].4

  • Errors of omission?  Check another one.

I devoted another whole post to Nissen and Wolski’s results depending upon the errors of multiple comparisons (multiplicity), in More Concerns With Nissen’s Avandia Analysis, January 29, 2014.  In brief, the more comparisons beyond one, the greater the bias toward finding a difference that does not really exist.  Nissen and Wolski reported eight.  Apparently, Dr. Nissen did not pay much attention to his own graph of the effect of multiple comparisons.

  • Errors due to multiplicity?  Check one more.

Interestingly, although Dr. Nissen makes much of pharmaceutical sponsorship as being sufficient to prove bias,5 his list did not mention bias from sponsorships.  However, he did claim that “Companies are directed to pay any honoraria, speaking or consulting fees directly to charity so that neither income nor tax deduction is received.”3  This claim is not what it appears to be.  As will be seen in an upcoming post, Dr. Nissen’s hypocrisy in this.  Apparently, he has never been supported by GlaxoSmithKline, but

“. . . Steven Nissen has acknowledged that his work is supported by many GlaxoSmithKline competitors including Pfizer, AstraZeneca, Sanofi-Aventis, Eli Lilly and the Japanese drug company called Takeda, which makes the diabetes drug Actos that competes directly with Avandia.”6

  • Sponsorship bias?   Add another.

While Dr. Nissen acknowledged limitations with the analysis, “his concern was mainly about the true magnitude of the risk, not whether or not the risk was present or absent.”2  Dr. Nissen did not mention it in his list,3 but it is bias, nonetheless.

  • Bias toward a particular outcome?   Add a very big one.

How important were these biases to the results?  Considering “the fragility of their findings,”7 even a small effect from bias would devastate Nissen and Wolski’s meta-analysis–without considering all of the other profound flaws in their study.

In commenting on the Nissen meta-analysis, Psaty and Furburg7 noted that the weaknesses of the study were substantial and that “a few events either way might have changed the findings for myocardial infarction or death from cardiovascular causes.  In this setting, the possibility that the findings were due to chance cannot be excluded.”1

Six different types of bias in just one study, four of which Dr. Nissen, himself, puts on a par with fraud, flaws, and misinterpretations.3

Maybe Dr. Nissen thinks that what is good for randomized controlled trials is too good for a meta-analysis1 that was merely based upon  randomized controlled trials.

Or, maybe he is just biased.

© 2014 Myron Shank, M.D., Ph.D.8

1 Nissen Steven E., Wolski Kathy.  Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes.  The New England Journal of Medicine 2007; 356:2457-2471.

2 Temple Robert.  Memorandum to Janet Woodcock:  Data on Rosiglitazone, August 8, 2010.  http://www.fda.gov/downloads/drugs/drugsafety/postmarketdrugsafetyinformationforpatientsandproviders/ucm226066.pdf

3 Steven Nissen.  Flaws, bias, misinterpretation and fraud
in randomized clinical trials.  http://spo.escardio.org/eslides/view.aspx?eevtid=48&fp=3132

4 Schachtman Nathan A.  Learning to embrace flawed evidence–the Avandia MDL’s Daubert opinion. Schachtman Law January 10th, 2011. http://schachtmanlaw.com/learning-to-embrace-flawed-evidence-the-avandia-mdls-daubert-opinion/

5 Sullivan Thomas.  Nissen relationships with industry:  CME, professional societies and red dresses.  Policy and Medicine March 18, 2010.  http://www.policymed.com/2010/03/nissen-relationships-with-industry-cme-professional-societies-and-red-dresses.html (The reader is strongly encouraged to read this revealing article about Steven Nissen’s character and credibility.)

6 Milloy Steven.  Junk science: diabetes drug scare or scam?  FoxNews.com June 10, 2007. http://www.foxnews.com/story/2007/06/10/junk-science-diabetes-drug-scare-or-scam/

7 Psaty Bruce M., Furberg Curt D. Rosiglitazone and cardiovascular risk.  New England Journal of Medicine 2007; 356:2522-2524.

8 I served:  in the speaker program for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, Chicago, Illinois, August 7, 1999; as a consultant for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, “Current Update and Discussion of the Glitazone Class of Oral Antidiabetics,” January 3, 2001; on Avandia Regional Advisory Panels in Cleveland, Ohio (February 4, 1999), Toledo, Ohio (September 15, 1999), San Juan, Puerto Rico (February 2000), and Beverly Hills, California (April 2000); as a preceptor for SmithKline Beechum, Lima, Ohio (June 23-25, 1999); and in a “Meet-the-Specialist,” SmithKline Beecham, Lima, Ohio, (2000). In 2000, SmithKline Beechum merged with GlaxoWellcome, to form GlaxoSmithKline.  Before it became impractical to continue prescribing rosiglitazone (Avandia, Avandamet, Avandaryl; GlaxoSmithKline), I prescribed both it and pioglitazone (ACTOS, ACTOSPlusMet, DuetAct; Takeda).

Strange Omissions in Nissen’s Avandia Analysis

In my post of January 27, 2014, Concerns at FDA:  Nissen’s Avandia Analysis, I discussed concerns expressed by Deputy Director Robert Temple, in a  memo to Janet Woodcock, Director of the Food and Drug Administration’s Center for Drug Evaluation and Research:  Nissen and Wolski’s2 peculiar and unblinded1 selective choices from those studies that qualified for their meta-analysis,3 their potential bias in doing the analysis,1 and their (improper) inclusion of DREAM (Diabetes REduction Assessment With ramipril And rosiglitazone Medication) after using it to develop their hypothesis.4  Dr. Temple concluded that “there was potential bias in its development (or at least the analysis was done with knowledge of what would emerge).”  Then, in my January 29, 2014 post, More Concerns With Nissen’s Avandia Analysis, I discussed some of Nissen and Wolski’s2 improper use of statistics related to their multiple comparisons between those who were treated with and those who were treated without rosiglitazone (Avandia; GlaxoSmithKline).

Nissen and Wolski did not consider the combined endpoint of acute myocardial infarctions plus cardiovascular deaths, attributing this to uncertainty about the possibility of counting the same event twice.2  However, for any reasonable assumption about the occurrence of both acute myocardial infarction and cardiovascular death in the same patient, the effect would be much less than the problem of multiple comparisons, which they ignored.

Speaking of multiple comparisons, Steven Nissen’s “Flaws, Bias, Misinterpretation and Fraud in Randomized Clinical Trials” showed that he was aware of the problem, but, although he used his own meta-analysis2 as an example to impugn another paper, Dr. Nissen hypocritically omited mention that he and Kathy Wolski2 were guilty of the very same multiple comparisons that he decried in others.5

“Stroke is not mentioned at all” by Nissen and Wolski, “a strange omission as almost all CV [cardiovascular] event trials include it.”1  Nissen and Wolski might have lacked the data, but, if so, they “apparently, did not seek it.”1  Ironically, Nissen and Wolski2 claim to have used Peto’s method in their meta-analysis,2 but “Peto, for example, has emphasized the need to find critical missing data (strokes, for example, or time of events) . . . .”1  Then again, as the Food and Drug Administration’s own analyses have shown, “rosiglitazone treatment was in fact associated with fewer strokes.”1

Since Nissen and Wolski did not believe that they had the data to combine acute myocardial infarctions and did not evaluate strokes, it is no surprise that they did not address major adverse cardiac events1 (acute myocardial infarctions plus strokes plus cardiovascular deaths), either.

Of course, acute myocardial infarctions can cause cardiovascular deaths, although not all cardiovascular deaths are caused by acute myocardial infarctions (Some, for example, are caused by strokes, which were curiously not even considered by Nissen and Wolski.).  Unless just one of these outcomes was counted per person, the number of independent observations (in statistical jargon, “n”) will also be exaggerated, giving us too much confidence in the apparent differences between groups.  Nissen and Wolski mentioned this problem, with respect to the measure, acute-myocardial-infarctions-plus-cardiovascular-deaths,2 but the same problem would also have affected the techniques of multiple stage statistical testing (See More Concerns With Nissen’s Avandia Analysis, January 29, 2014.), had actually used such techniques for their multiple comparisons.  Once again, however, for any reasonable assumptions, the potential magnitude of the error would have been far less than the errors introduced by not correcting for multiple comparisons.

The Food and Drug Administration’s own meta-analysis of fifty-two studies6 included all of the measures that Nissen and Wolski ought to have included.  In the meta-analysis of fifty-two studies, acute myocardial infarctions were “nominally statistically significant,”7 strokes trended “favorably for rosiglitazone,” neither cardiovascular nor all-cause mortality was even close to being “nominally statistically significant,”7 and the major adverse cardiovascular events category was not statistically significant.  Unlike Nissen and Wolski,2 The Food and Drug Administration at least acknowledged the impact of multiple comparisons:  “Any sort of correction for multiplicity leaves the finding well short of nominal statistical significance.”1,7

Nissen and Wolski’s2 “strange omissions” would have been bad enough, but there was plenty more.

© 2014 Myron Shank, M.D., Ph.D.8

1 Temple Robert.  Memorandum to Janet Woodcock:  Data on Rosiglitazone, August 8, 2010.  http://www.fda.gov/downloads/drugs/drugsafety/postmarketdrugsafetyinformationforpatientsandproviders/ucm226066.pdf

2 Nissen Steven E., Wolski Kathy.  Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes.  The New England Journal of Medicine 2007; 356:2457-2471.

3 Schachtman Nathan A.  Learning to embrace flawed evidence–the Avandia MDL’s Daubert opinion.  Schachtman Law January 10th, 2011. http://schachtmanlaw.com/learning-to-embrace-flawed-evidence-the-avandia-mdls-daubert-opinion/.

4 To review, I observed that “It is never appropriate to test a hypothesis with the same data that was used to generate it.”    For the statistically literate, I pointed out that excluding the statistical “degrees of freedom” that were improperly included from DREAM would, by itself, eliminate even the appearance of statistical significance.  I also noted that this was inexcusable from Dr. Nissen’s co-author, Kathy Wolski, since she is a statistician.

5 Steven Nissen.  Flaws, bias, misinterpretation and fraud in randomized clinical trials.  http://spo.escardio.org/eslides/view.aspx?eevtid=48&fp=3132

6 In other words, twenty-four percent more than the Nissen and Wolski meta-analysis.2

7 The phrases, “nominally statistically significant” and “nominal statistical significance” may refer to a “p” value of less than 0.05 or to the fact that an apparently statistically significant “p” value does not take into consideration complicating factors, such as multiple comparisons.  In the source memo, it is not always clear from the context whether the former, the later, or both meanings were intended.

I served:  in the speaker program for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, Chicago, Illinois, August 7, 1999; as a consultant for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, “Current Update and Discussion of the Glitazone Class of Oral Antidiabetics,” January 3, 2001; on Avandia Regional Advisory Panels in Cleveland, Ohio (February 4, 1999), Toledo, Ohio (September 15, 1999), San Juan, Puerto Rico (February 2000), and Beverly Hills, California (April 2000); as a preceptor for SmithKline Beechum, Lima, Ohio (June 23-25, 1999); and in a “Meet-the-Specialist,” SmithKline Beecham, Lima, Ohio, (2000).  In 2000, SmithKline Beechum merged with GlaxoWellcome, to form GlaxoSmithKline.  Before it became impractical to continue prescribing rosiglitazone (Avandia, Avandamet, Avandaryl; GlaxoSmithKline), I prescribed both it and pioglitazone (ACTOS, ACTOSPlusMet, DuetAct; Takeda).

More Concerns With Nissen’s Avandia Analysis

The patron saint of meta-analysis, Richard Peto, who helped bring the approach back from what we used to dismiss as ‘unplanned pooling,’ stressed approaches that would avoid bias and error, including selection of endpoints before the trials are analyzed and a reasonably high level of statistical significance.1

Nissen and Wolski said they used the Peto method, for their original meta-analysis of rosiglitazone (Avandia; GlaxoSmithKline).2  So, how did they measure up?

As noted in my January 27, 2014 post, Concerns at FDA:  Nissen’s Avandia Analysis, Deputy Director Robert Temple, in a  memo to Janet Woodcock, Director of the Food and Drug Administration’s Center for Drug Evaluation and Research, was critical of Nissen and Wolski’s2 peculiar and unblinded1 selective choices of studies (from among those that met their inclusion criteria).3  Dr. Temple also raised questions about the (improper) inclusion in their meta-analysis of the DREAM study, on which their hypothesis had been based.4  About the Nissen and Wolski meta-analysis, Dr. Temple concluded that “there was potential bias in its development (or at least the analysis was done with knowledge of what would emerge).”1

Nissen and Wolski reported doing eight different comparisons in their original meta-analysis.2  If these eight comparisons had been independent of (unrelated to) each other, the probability that they would have found a difference by chance, where none existed, would really have been 0.34,5 instead of the 0.05 that they claimed.2   In other words, instead of the probability of erroneously concluding, by chance, that there were differences between treatment with and without rosiglitazone being one-in-twenty, as they implied, the probability  would be more than one-in-three.

Unfortunately, acute myocardial infarctions and cardiovascular deaths are linked; whatever factors affect one, affect the other. “If the comparisons are not independent, it really is impossible to compute the probability . . . .”6  The truth lies somewhere between “p”=0.05 and “p”=0.34–a useless result.

One common technique is to make the threshold for statistical significance much stricter, adjusting  by dividing the required “p” value for significance by the number of comparisons made–in this case, 0.05/8=0.00625.7  This is the Bonferroni adjustment8  The Bonferroni adjustment is approximate and only appropriate if the comparisons are independent of each other.9

Some other techniques only test the individual components if the over-all comparison is significant–in other words, if (and only if) we know that there is a difference, we are entitled to find out where it is.  This is known as a multiple-stage test.10

There are other techniques, each of which has its own uses and limitations.10

Nissen and Wolski used none of these.

Return for concerns about what the Nissen and Wolski meta-analysis2 left out.

© 2014 Myron Shank, M.D., Ph.D.11

1 Temple Robert.  Memorandum to Janet Woodcock:  Data on Rosiglitazone, August 8, 2010.  http://www.fda.gov/downloads/drugs/drugsafety/postmarketdrugsafetyinformationforpatientsandproviders/ucm226066.pdf

2 Nissen Steven E., Wolski Kathy.  Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes.  The New England Journal of Medicine 2007; 356:2457-2471.

3 Schachtman Nathan A.  Learning to embrace flawed evidence–the Avandia MDL’s Daubert opinion.  Schachtman Law January 10th, 2011. http://schachtmanlaw.com/learning-to-embrace-flawed-evidence-the-avandia-mdls-daubert-opinion/.

4 As I pointed out, “It is never appropriate to test a hypothesis with the same data that was used to generate it.”  I also noted that this was inexcusable behavior from a statistician, such as Dr. Nissen’s co-author, Kathy Wolski.  For the statistically literate, I pointed out that excluding DREAM’s statistical “degrees of freedom,” which were improperly included, would have been sufficient, by itself, to have eliminated any pretense of statistical significance for Nissen and Wolski’s meta-analysis.

5 1.00-0.958=0.34.

6 Anonymous. The multiple comparisons problem. GraphPad Statistics Guide http://www.graphpad.com/guides/prism/6/statistics/index.htm?stat_the_problem_of_multiple_compar.htm. Accessed January 26, 2014.

7 The exact calculation is 1.00-0.95-8=0.0064 (Notice the negative sign on the exponent; this is the 8th-root of 0.95.).

8 Anonymous.  Glossary of statistical terms:  “Bonferroni adjustment,” The Institute for Statistics Education.   http://www.statistics.com/index.php?page=glossary&term_id=611. Accessed January 26, 2014.11 Again, this is inexcusable for a statistician, such as Kathy Wolski.

9 Anonymous.  Bonferroni.  Simple Interactive Statistical Analysis  http://www.quantitativeskills.com/sisa/calculations/bonhlp.htm>. Accessed January 26, 2014.

10 Anonymous. The GLM procedure:  multiple comparisons.  SAS/STAT User’s Guide. http://v8doc.sas.com/sashtml/stat/chap30/sect35.htm.  Accessed January 26, 2014.

11  I served:  in the speaker program for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, Chicago, Illinois, August 7, 1999; as a consultant for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, “Current Update and Discussion of the Glitazone Class of Oral Antidiabetics,” January 3, 2001; on Avandia Regional Advisory Panels in Cleveland, Ohio (February 4, 1999), Toledo, Ohio (September 15, 1999), San Juan, Puerto Rico (February 2000), and Beverly Hills, California (April 2000); as a preceptor for SmithKline Beechum, Lima, Ohio (June 23-25, 1999); and in a “Meet-the-Specialist,” SmithKline Beecham, Lima, Ohio, (2000).  In 2000, SmithKline Beechum merged with GlaxoWellcome, to form GlaxoSmithKline.  Before it became impractical to continue prescribing rosiglitazone (Avandia, Avandamet, Avandaryl; GlaxoSmithKline), I prescribed both it and pioglitazone (ACTOS, ACTOSPlusMet, DuetAct; Takeda).

Concerns at FDA: Nissen’s Avandia Analysis

In a memo to Janet Woodcock, Director of the Food and Drug Administration’s Center for Drug Evaluation and Research, Deputy Director Robert Temple, pointed out that “it is MACE [major adverse cardiovascular events] that seems like the most informative endpoint and it is the one regularly used to assess favorable CV [cardiovascular] effects of drugs and drug risks (e.g., in the diabetes guidance),”1 rather than the individual components (myocardial infarction, strokes, and deaths due to the other two).1  As we will see, however, the individual components “were critical in the Nissan meta-analysis”1 of rosiglitazone (Avandia; GlaxoSmithKline).

As Dr. Temple pointed out, the Food and Drug Administration’s unfavorable 2010 review of the safety of rosiglitazone “did not really consider the weight of the various lines of evidence in much detail nor did it focus on the particular findings that were most critical”1 in light of Nissen and Wolski’s original meta-analysis.2

What were his concerns?

To begin with, why did Nissen and Wolski pool the large, long-term, ADOPT (A Diabetes Outcome Progression Trial) and DREAM (Diabetes REduction Assessment With ramipril And rosiglitazone Medication) studies with “40 relatively small studies, most of them 6 months or shorter”?1  As Dr. Temple noted, this choice had “important consequences to the outcome” of the meta-analysis,  but was “not fully explained in the paper.”1

Dr. Nissen makes much of the fact that RECORD (Rosiglitazone Evaluated for Cardiovascular Outcomes and Regulation of Glycemia in Diabetes) was not blinded.   However, he knew the results of ADOPT and DREAM, “so that the effect of combining these studies with the smaller ones was clear.”1  Nissen and Wolski picked and chose among studies which met their inclusion criteria, with full knowledge of what effects these choices would have on the results of their meta-analysis,2 and “it must be recognized that there was potential bias in its development (or at least the analysis was done with knowledge of what would emerge)” by doing so.1

Not everyone is so diplomatic.

There were, in fact, studies available to Nissen and to the public, which met Nissen’s inclusion criteria, but which he failed to include in his meta-analysis.  Nissen’s meta-analysis was thus biased by its failure to have conducted a complete, thorough review of the medical literature for qualifying RCTs [randomized controlled trials].3

In fact, according to Dr. Temple, Dr. Nissen has attributed his interest in rosiglitazone’s impact on acute myocardial infarctions on the six extra myocardial infarctions among patients treated with rosiglitazone in DREAM.1  “In any event, speculation aside, this was not a blinded effort and the results were obtained with full knowledge of the effect of various analytic decisions on the outcome by the analysts, a very common situation with meta-analyses.”1

“As noted above, in his presentation at the July 2010 AC [advisory committee] meeting, Nissen made it clear that it was DREAM, with its adverse trends on AMI [acute myocardial infarction] as well as other endpoints (especially CHF [congestive heart failure]), that provoked the meta-analysis.  In such a case one could ask whether DREAM should have been included in the meta-analysis.1

One could ask, but the question would be merely rhetorical.  It is never appropriate to test a hypothesis with the same data that was used to generate it (For the statistically literate, I would point out that just excluding DREAM’s statistical “degrees of freedom,” which were improperly included in Nissen and Wolski’s meta-analysis, would have been sufficient, by itself, to eliminate any pretense of statistical significance.).  Maybe cardiologist Steven Nissen, M.D., can be excused for this breach of fundamental statistical principles, but his co-author, statistician Kathy Wolski, cannot be.4

As Dr. Temple observes, “That is, of course, not what we expect in designing and evaluating RCTs [randomized controlled trials], where we stress the need to make all analytic plans before results are in hand to avoid bias.”1  He continues:  “I recognize the irony of this:  the not very precise, not well-specified findings in the [Nissen and Wolski] meta-analysis are held to a lower standard than the later trials that seek to examine the concerns stimulated by the [Nissen and Wolski] meta-analysis.”1

Nissen and Wolski’s meta-analysis was slip-shod, unblinded, biased, and inconsistent.  To defend its dubious conclusions, Dr. Nissen and his allies have had to resort to double standards.

There is more to come.

© 2014 Myron Shank, M.D., Ph.D.5

1 Temple Robert.  Memorandum to Janet Woodcock:  Data on Rosiglitazone, August 8, 2010.  http://www.fda.gov/downloads/drugs/drugsafety/postmarketdrugsafetyinformationforpatientsandproviders/ucm226066.pdf

2 Nissen Steven E., Wolski Kathy.  Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes.  The New England Journal of Medicine 2007; 356:2457-2471.

3 Schachtman Nathan A.  Learning to embrace flawed evidence–the Avandia MDL’s Daubert opinion.  Schachtman Law January 10th, 2011. http://schachtmanlaw.com/learning-to-embrace-flawed-evidence-the-avandia-mdls-daubert-opinion/.

4Others have found the quality of her work seriously wanting, as well (Duckworth William, Abraira Carlos, Moritz Thomas, Reda Domenic, Emanuele Nicholas, Reaven Peter D., Zieve Franklin J., Marks Jennifer, Davis Stephen N., Hayward Rodney, Warren Stuart R., Goldman Steven, McCarren Madeline, Vitek Mary Ellen, Henderson William G., Huang Grant D., for the VADT Investigators. Glucose control and vascular complications in veterans with type 2 diabetes. New England Journal of Medicine 2009; 360:129-139. See also my posts:  Steven Nissen and The Not-So-Great Avandia Controversy, December 18, 2013; Nissen, Wolski, and How Not to Do Meta-Analyses, December 19, 2013; “Where’s the Beef?” for Avandia’s Risk?, January 13, 2014; Nissen and Wolski’s Avandia “Significance”, January 15, 2014; and “But wait! There’s more!” The RECORD on Avandia, January 17, 2014.).  In particular, even though her own “updated” meta-analysis with Dr. Nissen was statistically non-significant, she was not deterred from claiming otherwise (Nissen Steven E., Wolski Kathy. Rosiglitazone revisited: an updated meta-analysis of risk for myocardial infarction and cardiovascular mortality. Archives of Internal Medicine 2010; 170:1191-1201.).

5 I served:  in the speaker program for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, Chicago, Illinois, August 7, 1999; as a consultant for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, “Current Update and Discussion of the Glitazone Class of Oral Antidiabetics,” January 3, 2001; on Avandia Regional Advisory Panels in Cleveland, Ohio (February 4, 1999), Toledo, Ohio (September 15, 1999), San Juan, Puerto Rico (February 2000), and Beverly Hills, California (April 2000); as a preceptor for SmithKline Beechum, Lima, Ohio (June 23-25, 1999); and in a “Meet-the-Specialist,” SmithKline Beecham, Lima, Ohio, (2000).  In 2000, SmithKline Beechum merged with GlaxoWellcome, to form GlaxoSmithKline.  Before it became impractical to continue prescribing rosiglitazone (Avandia, Avandamet, Avandaryl; GlaxoSmithKline), I prescribed both it and pioglitazone (ACTOS, ACTOSPlusMet, DuetAct; Takeda).

Are Avandia Advantages in VADT “Statistical Nonsense” (Nissen)?

Steven Nissen, M.D., “scoffed at the analysis” of the Veterans Administration Diabetes Trial which showed statistically “significantly reduced rates of cardiovascular events with rosiglitazone,”1 before he even knew how it was done (See my post, Steven Nissen Calls the Kettle “Black”–VADT’s Avandia Analysis, January 22, 2014.).

So how were the Veterans Administration Diabetes Trial analyses of the safety of rosiglitazone  (Avandia; GlaxoSmithKline) conducted, and what did they show?

There were three types of analyses2 (which I will explain as simply as I can):

  • Case-control analysis
  • Time-dependent covariate survival analysis
  • Propensity matching analysis

Case-Control Analysis

In a case-control analysis, patients who have a particular event (cases) are matched to those who do not have that event (controls).

In this analysis of the Veterans Administration Diabetes Trial, the event categories were myocardial infarctions (heart attacks), strokes, death due to either of these causes, or major adverse cardiovascular events (any of the other events).  Cases and controls were matched on prior history of myocardial infarctions and strokes, use of insulin at the beginning of the study, time to the event (up to the end of the study, for the controls), age, total cholesterol, and at least one other characteristic:  duration of diabetes, obesity as measured by body mass index, systolic blood pressure, diastolic blood pressure, HDL (“good”) cholesterol, or HbA1c (an estimate of average blood glucose).  The cases and controls were then compared on the number of visits for which rosiglitazone was prescribed and the average dose of rosiglitazone at each of those visits.2

For each of the four categories, the patients who had the “events” were prescribed rosiglitazone at fewer visits and at lower doses than those who did not have the “events.”2  While these results were not statistically significant (they could easily have occurred by chance), and while both the “events”2 and the lower doses and duration of rosiglitzone therapy might have been related due to episodes of hypoglycemia (low blood sugar), it is still telling that the results were opposite in direction to those predicted from Nissen and Wolski’s meta-analyses.3,4

Time-Dependent Covariate Analysis

In this analysis, the effect of rosiglitzone was examined with respect to the time until the first “event,” where the event categories were the same as for the case-control analysis, above.2  Time-dependent covariate analysis allows evaluation of the effects of variables whose values change with time,2 unlike the case-control analysis, above.

For both the 4 and the 8 mg doses of rosiglitazone, the risk ratio5 for first events was lower, regardless of whether or not the data was adjusted for patient characteristics at entry into the study (age, use of insulin, systolic blood pressure, and HDL), or characteristics both at entry into the study and throughout the study (age, use of insulin at entry, HDL, and HbA1c).2

For myocardial infarctions with the 4 mg dose, only the unadjusted risk ratio was statistically significant (in other words, unlikely to have occurred by chance alone), but with the 8 mg dose, the only the risk ratio that was not statistically significant was the one adjusted for characteristics both at entry and throughout the study.  For strokes, cardiovascular deaths, and major adverse cardiovascular events, the risk ratios were statistically significantly lower with both doses of rosiglitazone, regardless of adjustment or non-adjustment.2  The higher the dose (0, 4, or 8 mg), the better rosiglitazone looked–not what one would expect from an unacceptably risky drug.

Interestingly, in each case, the “adjusted” and “unadjusted” results were almost identical.2

Just as importantly as their statistical significances, the sizes of each of these apparent differences were clinically highly significant.2

Note that statistical significance of the overall comparison (major adverse cardiovascular events) was justified comparison within its subclasses, “protecting” them from being falsely identified as “significant” as an artifact (artificial result) of multiple comparisons.6

Propensity Analysis

“Propensity analysis” is just a hard way of saying that those who were always prescribed rosiglitazone were compared with those who were never prescribed rosiglitazone.

Overall, there were generally small, but important, differences in for these groups’ characteristics, upon entering the study.  Except for a nearly two-fold difference in smoking, these differences made the “events” less likely in those who were always treated with rosiglitazone than in those who never were.2

However, after matching them with a mathematical model (known as a “step-wise logistic regression”), the characteristics at entry into the study were almost identical between the two groups.2

Again, whether adjusted or not, the risk ratios for all four “events” were much lower in the group that had always been treated with rosiglitazone than in the group that never had been (The overall analysis, major adverse cardiovascular events, was statistically significant, justifying statistical testing of the subgroups, which were not significantly different6).  Once again, the “adjusted” and “unadjusted” results were almost identical.2

Conclusions

“Based on the results from the VADT study, there is no evidence to suggest that use of rosiglitazone increases the risk for cardiovascular mortality or morbidity in patients with type-2 diabetes.”7

Actually, for cardiovascular results (heart attacks, strokes, related deaths, and the combination of the other results) that were uniformly clinically, and often statistically, significant in rosiglitazone’s favor,7 that was a major understatement–especially when, unlike Nissen and Wolski’s meta-analyses,3,4 the results were fairly robust, being demonstrated with very different approaches to the data.7  More to the point, “If anything, rosiglitazone had a protective effect.”1

Of perhaps the greatest importance, severe hypoglycemia (blood sugar low enough to require assistance from someone else) powerfully predicted cardiovascular events.1  Treating with higher doses, or with additional drugs, to lower the blood sugar will almost always increase the risk of hypoglycemia–which may be the “real” cause of any apparent increase in cardiovascular events from a diabetes drug.  This may help to explain the general lack of benefit from “tight” blood glucose control seen in the Veterans Administration Diabetes Trial7 and multiple other studies.  It might even be relevant to Nissen and Wolski’s own meta-analyses!3,4

No one claims that these unplanned analyses of the Veterans Administration Diabetes Trial are ideal.  They were not even published with the other results.7  However, despite the limitations of these analyses, it is clear that the chief statistician for the Veterans Administration Diabetes Trial, Thomas E. Moritz, M.S., actually knows statistics,7 unlike heart doctor, Steven E. Nissen, M.D.8

“Statistical nonsense,” Dr. Nissen?  It is time for you to stop calling kettles black.

© 2014 Myron Shank, M.D., Ph.D.9

1 Gever John.  ADA:  VA Diabetes Trial appears to vindicate rosiglitazone (Avandia) safety.   MedPage Today June 8, 2008.   http://www.medpagetoday.com/MeetingCoverage/ADA/9749

2 Moritz Thomas E.  Impact of the use of rosiglitazone in VADT.  http://www.fda.gov/downloads/AdvisoryCommittees/CommitteesMeetingMaterials/Drugs/EndocrinologicandMetabolicDrugsAdvisoryCommittee/UCM224743.pdf

3 Nissen Steven E., Wolski Kathy.  Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes.  The New England Journal of Medicine 2007; 356:2457-2471.

4 Nissen Steven E., Wolski Kathy.  Rosiglitazone revisited:  an updated meta-analysis of risk for myocardial infarction and cardiovascular mortality.  Archives of Internal Medicine 2010; 170:1191-1201.

5 See my post: What is the Risk of Avandia?, December 23, 2013.

6 See my post: Nissen, Wolski, and How Not to Do Meta-Analyses, December 19, 2013.

7 Duckworth William, Abraira Carlos, Moritz Thomas, Reda Domenic, Emanuele Nicholas, Reaven Peter D., Zieve Franklin J., Marks Jennifer, Davis Stephen N., Hayward Rodney, Warren Stuart R., Goldman Steven, McCarren Madeline, Vitek Mary Ellen, Henderson William G., Huang Grant D., for the VADT Investigators.  Glucose control and vascular complications in veterans with type 2 diabetes.  New England Journal of Medicine 2009; 360:129-139.

8 While Kathy Wolski, Dr. Nissen’s co-author for both his original5 and his updated6 rosiglitazone meta-analyses, is a statistician, not only is her opinion of the statistical analysis for the Veterans Administration Diabetes Trial not available, but the quality of her own work has been found seriously wanting7  (See also my posts:  “Where’s the Beef?” for Avandia’s Risk?, January 13, 2014; Nissen and Wolski’s Avandia “Significance”, January 15, 2014; and “But wait! There’s more!” The RECORD on Avandia, January 17, 2014.).  In particular, even though her own “updated” meta-analysis with Dr. Nissen6 was statistically non-significant,7 she was not deterred from claiming otherwise.  While Dr. Nissen could (but will not) claim ignorance, she is a statistician and should be held to a higher standard.  (For those who are wondering, yes, I have studied statistics, at both the collegiate and graduate levels.)

9 I served:  in the speaker program for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, Chicago, Illinois, August 7, 1999; as a consultant for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, “Current Update and Discussion of the Glitazone Class of Oral Antidiabetics,” January 3, 2001; on Avandia Regional Advisory Panels in Cleveland, Ohio (February 4, 1999), Toledo, Ohio (September 15, 1999), San Juan, Puerto Rico (February 2000), and Beverly Hills, California (April 2000); as a preceptor for SmithKline Beechum, Lima, Ohio (June 23-25, 1999); and in a “Meet-the-Specialist,” SmithKline Beecham, Lima, Ohio, (2000).  In 2000, SmithKline Beechum merged with GlaxoWellcome, to form GlaxoSmithKline.  Before prescribing rosiglitazone (Avandia, Avandamet, Avandaryl; GlaxoSmithKline) became impractical (and then restricted), I prescribed both it and pioglitazone (ACTOS, ACTOSPlusMet, DuetAct; Takeda).

Steven Nissen Calls the Kettle “Black” (VADT’s Avandia Analyses)

Nissen’s Prescience of the Veterans Administration Diabetes Trial

Without ever having read the (at that time) unpublished results of the Veterans Administration Diabetes Trial (VADT),  “Dr. Nissen scoffed at the analyses” of safety of rosiglitazone (Avandia, GlaxoSmithKline) before it had even been presented.1

“The VADT trial provides no useful insights into the cardiovascular safety of rosiglitazone,” Dr. Nissen said in an email.

“Approximately 80% of patients in both the intensive and standard treatment groups received rosiglitazone [Avandia; GlaxoSmithKline].2  Therefore, a statistical analysis of the safety of this drug is not possible from VADT,” he said.  “Any assertion that VADT demonstrates the safety of rosiglitazone is statistical nonsense.”1

Oops, Dr. Nissen!

Actually, it is quite possible, inspite of Dr. Nissen’s statistical pontifications.

The Veterans Administration Diabetes Trial was designed to evaluate the effects of intensive and conventional glucose control for diabetes,3 not to compare drug regimens.4  However, after Nissen and Wolski’s original meta-analysis,5 the investigators “agreed to conduct” the “specific analysis of events with rosiglitazone” with their data.1

Dr. Nissen “scoffed at the analyses Moritz reported,” which showed statistically “significantly reduced rates of cardiovascular events with rosiglitazone.”  Simply comparing the intensive with the conventional glucose control arms would have revealed nothing about rosiglitazone, since most of the patients in each arm received that drug at some point, but that is not what Mr. Moritz did.4

In my next post, I will explain how the Veterans Administration Diabetes Trial analyses of rosiglitazone’s safety were conducted and what they showed.

© 2014 Myron Shank, M.D., Ph.D.6

1 Gever John.  ADA:  VA Diabetes Trial appears to vindicate rosiglitazone (Avandia) safety.   MedPage Today June 8, 2008.   http://www.medpagetoday.com/MeetingCoverage/ADA/9749

2 I do not know how Dr. Nissen came up with these numbers.  “Almost every VADT patient received rosiglitazone sometime during the study.”3 The percentages steadily trended downward from approximately 90% in both groups, at the beginning of the study, to about 25 and 30% in the conventionally- and intensively-treated groups, respectively, by the study’s end.3  Apparently, there no time when the percentage approximated 80% in both groups, and, during most of the study, it was less than that in both groups.3

3 Moritz Thomas E.  Impact of the use of rosiglitazone in VADT.  http://www.fda.gov/downloads/AdvisoryCommittees/CommitteesMeetingMaterials/Drugs/EndocrinologicandMetabolicDrugsAdvisoryCommittee/UCM224743.pdf

4 Duckworth William, Abraira Carlos, Moritz Thomas, Reda Domenic, Emanuele Nicholas, Reaven Peter D., Zieve Franklin J., Marks Jennifer, Davis Stephen N., Hayward Rodney, Warren Stuart R., Goldman Steven, McCarren Madeline, Vitek Mary Ellen, Henderson William G., Huang Grant D., for the VADT Investigators.  Glucose control and vascular complications in veterans with type 2 diabetes.  New England Journal of Medicine 2009; 360:129-139.

5 Nissen Steven E., Wolski Kathy.  Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes.  The New England Journal of Medicine 2007; 356:2457-2471.

6 I served:  in the speaker program for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, Chicago, Illinois, August 7, 1999; as a consultant for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, “Current Update and Discussion of the Glitazone Class of Oral Antidiabetics,” January 3, 2001; on Avandia Regional Advisory Panels in Cleveland, Ohio (February 4, 1999), Toledo, Ohio (September 15, 1999), San Juan, Puerto Rico (February 2000), and Beverly Hills, California (April 2000); as a preceptor for SmithKline Beechum, Lima, Ohio (June 23-25, 1999); and in a “Meet-the-Specialist,” SmithKline Beecham, Lima, Ohio, (2000).  In 2000, SmithKline Beechum merged with GlaxoWellcome, to form GlaxoSmithKline.  Before prescribing rosiglitazone (Avandia, Avandamet, Avandaryl; GlaxoSmithKline) became impractical (and then restricted), I prescribed both it and pioglitazone (ACTOS, ACTOSPlusMet, DuetAct; Takeda).

Dr. Nissen: “Just Say No” to Learning About Avandia

As Steven Nissen, M.D., sees it, the only reason that the Food and Drug Administration followed through with its long-announced promise for an independent “re-adjudication” (re-judging) of the RECORD1 study was that “the leadership of the division of the FDA responsible for drug regulation, the Center for Drug Evaluation and Research (CDER), is seeking to avoid accountability for its role in the Avandia tragedy.”2

Dr. Nissen requested time to present his analyses about rosiglitazone (Avandia, Avandamet, Avandaryl; GlaxoSmithKline) to the Food and Drug Administration, but when he told them what he had to say, they concluded that it would merely duplicate other presentations.3

Uh, oh.  Dr. Nissen is not happy.  According to him, “the meeting seems systematically designed to absolve the drug of harm and the CDER leadership of any responsibility for ignoring the public health hazard of Avandia.  The current effort is intended to ‘whitewash’ the Avandia scandal and re-write history.”  He alleges that this is because “The leadership of CDER was intensely embarrassed by these revelations and furious with us for publicly challenging the safety of Avandia (and indirectly the competence and integrity of CDER).”2

Really?  And we are to believe that the same Dr. Nissen whose meta-analyses, “history” shows, have not only failed scientific and statistical scrutiny,4 but, even worse, have not been supported by real studies5 (let alone his own “updated” meta-analysis6) was not the least bit fearful of further embarrassment7 from the RECORD data?  Are we also to believe that the man who worked for Takeda, maker of the rival pioglitazone (ACTOS, ACTOSPlusMet, DuetAct), but not for GlaxoSmithKline, manufacturer of  rosiglitazone (Avandia, Avandamet, Avandaryl), was not above attempting to “blackwash” his opponents?  Are we not to even consider that this Dr. Nissen, who has been so strident in his attacks on anyone who does not support him, might be blackmailing those who are reluctant to have him study their drugs with his intravascular (within blood vessels) ultrasound procedure?8

Remember, Dr. Nissen, that ““to censor public debate about a critical public health issue is unacceptable and represents a grave threat to academic freedom.”9  You said that.  Those are your own words.  You did not believe that they meant allowing the TIDE study to continue–and possibly embarrass you.10  Clearly, you did not believe that they meant that the Food and Drug Administration should keep its word and allow the RECORD study to be independently re-adjudicated–which did embarrass you.5  Did you protest when the Chief of the Department of Medicine gave the endocrinologists in your own Cleveland Clinic strict orders not to say a word about rosiglitazone, with either an explicit or implicit threat that exercise of their “academic freedom” would cost them their jobs11–or were you afraid of being embarrassed?

Dr. Nissen, we have heard you–over and over again.  Until you have nothing new to say, you should do something new and say nothing.

© 2014 Myron Shank, M.D., Ph.D.

1 Rosiglitazone Evaluated for Cardiac Outcomes and Regulation of Glycaemia in Diabetes

2 Nissen Steven E.  Steven Nissen:  the hidden agenda behind the FDA’s new Avandia hearings. Forbes Pharma & Healthcare, May 23, 2013.  http://www.forbes.com/sites/matthewherper/2013/05/23/steven-nissen-the-hidden-agenda-behind-the-fdas-avandia-hearings/

3 Herper Matthew. The FDA responds to Steve Nissen’s criticism of upcoming Avandia meeting. Forbes Pharma & Healthcare, May 23, 2013.  http://www.forbes.com/sites/matthewherper/2013/05/23/the-fda-responds-to-steve-nissens-criticism-of-upcoming-avandia-meeting/

4 See my posts:  Steven Nissen and The Not-So-Great Avandia Controversy, December 18, 2013 http://drshank.wordpress.com/2013/12/18/steven-nissen-and-the-not-so-great-avandia-controversy/; Nissen, Wolski, and How Not to Do Meta-Analyses, December 19, 2013 http://drshank.wordpress.com/2013/12/19/nissen-wolsky-and-how-not-to-do-a-meta-analysis-2/; What is the Risk of Avandia?, December 23, 2013 http://drshank.wordpress.com/2013/12/23/what-is-the-risk-of-avandia/; “Where’s the Beef?” for Avandia’s Risk?, January 13, 2014 http://drshank.wordpress.com/2014/01/13/wheres-the-beef-for-avandias-risk/; Nissen and Wolski’s Avandia “Significance,” January 15, 2014 http://drshank.wordpress.com/2014/01/15/nissen-and-wolskis-avandia-significance/.  (For those who are wondering, yes, I have studied statistics at both the collegiate and graduate levels.)

5 See my posts:  ADOPT the DREAM of a RECORD for Avandia, December 26, 2013 http://drshank.wordpress.com/2013/12/26/adopt-the-dream-of-a-record-for-avandia/ and “But wait! There’s more!” The RECORD on Avandia, January 17, 2014 http://drshank.wordpress.com/2014/01/17/but-wait-theres-more/

6 Nissen Steven E., Wolski Kathy.  Rosiglitazone revisited: an updated meta-analysis of risk for myocardial infarction and cardiovascular mortality.  Archives of Internal Medicine 2010; 170:1191-1201.

7 “Nevertheless, the results of RECORD do not substantiate the findings from the Nissen/Wolski meta-analysis on myocardial infarction and cardiovascular death.”  Furthermore, there was a “favorable trend for rosiglitazone on all-cause mortality.”  (Unger Ellis F.  FDA cardiologist’s perspective on RECORD:  joint meeting of the Endocrinologic and Metabolic Drugs Advisory Committee and the Drug Safety and Risk Management Advisory Committee, Avandia® (rosiglitazone) – July 13-14, 2010. http://www.fda.gov/downloads/advisorycommittees/committeesmeetingmaterials/drugs/endocrinologicandmetabolicdrugsadvisorycommittee/ucm218485.pdf)

8 The story behind Dr. Nissen’s claims to have all payments from drug companies go to charities will have to wait for another post.  In the meantime, I disclose that I served:  in the speaker program for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, Chicago, Illinois, August 7, 1999; as a consultant for Takeda Pharmaceuticals America, Inc. and Eli Lilly and Company, “Current Update and Discussion of the Glitazone Class of Oral Antidiabetics,” January 3, 2001; on Avandia Regional Advisory Panels in Cleveland, Ohio (February 4, 1999), Toledo, Ohio (September 15, 1999), San Juan, Puerto Rico (February 2000), and Beverly Hills, California (April 2000); as a preceptor for SmithKline Beechum, Lima, Ohio (June 23-25, 1999); and in a “Meet-the-Specialist,” SmithKline Beecham, Lima, Ohio, (2000).  In 2000, SmithKline Beechum merged with GlaxoWellcome, to form GlaxoSmithKline.  Before it became impractical to continue prescribing rosiglitazone (Avandia, Avandamet, Avandaryl; GlaxoSmithKline), I prescribed both it and pioglitazone (ACTOS, ACTOSPlusMet, DuetAct; Takeda).

9 Husten Larry.  EHJ editors rebuffed GSK efforts to suppress Nissen editorial on rosiglitazone.  Cardiobrief April 30, 2010.  http://cardiobrief.org/2010/04/30/ehj-editors-rebuffed-gsk-efforts-to-suppress-nissen-editorial-on-rosiglitazone/.

10 See my post:  Avandia’s TIDE Is Stopped, December 24, 2013.  http://drshank.wordpress.com/2013/12/24/avandias-tide-goes-out/

11 I know several endocrinologists from the Cleveland Clinic Foundation.  After Dr. Nissen’s original meta-analysis came out, every one of them refused to say so much as a word about rosiglitazone, explaining that their department chairman had emphatically prohibited them from doing so (One acknowledged that their chairman had ordered them to stop using the drug.).  It is obvious why their chairman would forbid criticism of a powerful member of the institution; it is equally obvious that there would be no reason to prevent the endocrinologists from speaking out (or from continuing to use rosiglitazone), if they had supported Dr. Nissen.  Not one endocrinologist I know (and I have represented one out of every nine in the United States) supported Dr. Nissen, and many (including myself) have been outspoken critics.