Avandia’s TIDE Is Stopped

Remember TIDE (Thiazolidinedione Intervention with Vitamin D Evaluation), the study that was designed to provide definitive results about the risk–or lack thereof–for myocardial infarctions and cardiovascular death from rosiglitazone (Avandia, Avandamet, Avandaryl; GlaxoSmithKline)?  It got underway in 2009.¹

Enrollment of patients was sluggish, possibly because of the adverse publicity about rosiglitazone’s safety.²  Food and Drug Administration (FDA) epidemiologists David Graham and Kate Gelperin, arguing that GlaxoSmithKline had never proved that rosiglitazone was equivalent to pioglitazone (ACTOS, Takeda), called TIDE “unethical and exploitative.”²  Of course, this was disingenuous, since Drs. Graham and Gelperin knew full well both that TIDE was designed to prove that rosiglitazone was equivalent to pioglitazone and that they had newly minted the “standard” on which they were basing their arguments.  But even equivalence would not have been enough for Drs. Graham and Gelperin:  “That is, there must be equal evidence favoring both therapies.  But that is not the case here because no one has argued that rosiglitazone is safer than or preferable to pioglitazone.”²  So, Drs. Graham and Gelperin, is the standard equivalence, is it superiority, or is it flexible (whatever it takes for you to eliminate rosiglitazone)?

“Given the weight of evidence regarding cardiovascular risks with rosiglitazone, we believe that TIDE is an unethical study and that it was unethical before it was started,” opined Drs. Graham and Gelperin.  On the contrary, even then, the “weight” of the evidence against rosiglitazone was highly controversial,  I would argue that, ethically, head-to-head comparisons ought to be the norm.  Without them, drug choices, whether at the regulatory or the clinical level, are arbitrary opinions, rather than evidence-based decisions.  I would particularly argue that ethics required a study comparing rosiglitazone and pioglitazone, such as TIDE.

“In our view,” Drs. Graham and Gelperin continued, “the TIDE trial is unethical because it subjects human beings to unnecessary risks without any possibility of a meaningful, unique health benefit from rosiglitazone.”²  In the first place, such a question can only be answered by a direct comparison, such as TIDE, and in the second place, the standard for drugs is “safe and effective” for the intended use, not “meaningful, unique health benefits.”

“Patients bear the full burden of risk of AMI [acute myocardial infarction, “heart attack”], heart failure, and death, for the purpose of establishing with definitive certainty that rosiglitazone increases cardiovascular risk (the null hypothesis for the non-inferiority components of this trial), with no likely expectation of unique benefit.”  Really?  By definition, a “null hypothesis” is a hypothesis that the effect or difference is null–nothing.  To establish “that rosiglitazone increases cardiovascular risk” would require rejecting the null hypothesis of equivalency.  Obviously, these two either do not know what a “null hypothesis” is, or they are being dishonest.  Either way, they are discredited.

“Is it ethical to enroll patients in a clinical trial where the goal is to prove harm?” Dr. Graham asked during last week’s joint meeting of the Endocrinology and Metabolic Drugs Advisory Committee and the Drug Safety and Risk Management Advisory Committee.³  Of course, this begs the question (a logical fallacy) that the goal was to prove harm.  It was not.  On the contrary, the main goal of the study was to prove that rosiglitazone was no worse than placebo, and then that it was superior to placebo for myocardial infarctions and cardiovascular deaths; the secondary goal was to prove that rosiglitazone was no more than 20% worse than pioglitazone (which was considered to have neutral effects).¹  Drs. Graham and Gelperin should have known better; were they making things up to fit their agenda?

According to Dr. Graham, “The best” that participants could “hope for is to not get a drug that causes a problem.”³  Again, whether or not rosiglitazone causes problems, other than congestive heart failure–as pioglitazone does and as troglitazone (Rezulin, Warner-Lambert) did–can only be established by in a properly designed study, such as TIDE.

Drs. Graham and Gelperin’s criticism of the vitamin D component of the study was a red-herring–the same kind of distraction that they accused GlaxoSmithKline of using by including that part of the study.  While I believe that studies of vitamin D (actually, “hormone D”) need to be done, I agree that it was not relevant to a study of a drug that was under fire.  Instead of distracting from negative publicity about the risks of rosiglitazone, however, I believe that the vitamin D component was a distraction from conducting the essential rosiglitazone arm of the study.

In my opinion, it was the conduct of the likes of Drs. Graham and Gelperin that was unethical, not TIDE.  The only ethically acceptable justification for scuttling TIDE was the fact that improper criticisms had probably made continuation of the TIDE unfeasible.  Participants would have had to be informed that rosiglitazone might cause more cardiovascular events than pioglitazone, told about the Risk Evaluation and Mitigation Strategy use restrictions, and advised that they were unlikely to receive rosiglitazone outside of the study.³

The Food and Drug Administration had required GlaxoSmithKline to perform TIDE.  A majority of the members of the Advisory Committee and of the Office of New Drugs recommended continuing TIDE.   Janet Woodcock, M.D., Director of Center for Drug Evaluation and Research (CDER) over-ruled them, because of the restrictions that she had decided were necessary and of “the level of concern about its cardiovascular safety.”¹

TIDE was stopped in 2010.¹  What legend holds that King Canute had failed to do was now a fait accompli.  While Dr. Woodcock said that the Food and Drug Administration still required GlaxoSmithKline to conduct a study comparing rosiglitazone with pioglitazone “if feasible and appropriate,”¹ it is highly unlikely that this could be done, now.  Stopping TIDE virtually guaranteed that an answer about rosiglitazone will never be obtained and that opponents of rosiglitazone, such as Drs. Nissen, Graham, and Gelperin, will always be able to promote themselves by calling attention to uncertainties about the drug.

Maybe that was their goal.

© 2013 Myron Shank, M.D., Ph.D.

¹ Parks Mary H.  Introduction memorandum.  Readjudication of the Rosiglitazone Evaluated for Cardiovascular Outcomes and Regulation of Glycemia in Diabetes Trial (RECORD):  Joint Meeting of the Endocrinologic and Metabolic Drugs Advisory Committee and the Drug Safety and Risk Management Advisory Committee June 5—6, 2013. http://www.fda.gov/downloads/advisorycommittees/committeesmeetingmaterials/drugs/endocrinologicandmetabolicdrugsadvisorycommittee/ucm354859.pdf

² Graham David J., Gelperin Kate.  Comments on RECORD, TIDE, and the benefit-risk assessment of rosiglitazone vs. pioglitazone. In: FDA Briefing Document Advisory Committee Meeting for NDA 21071 Avandia (rosiglitazone maleate) tablet July 13 and 14, 2010.  http://www.fda.gov/downloads/AdvisoryCommittees/CommitteesMeetingMaterials/Drugs/EndocrinologicandMetabolicDrugsAdvisoryCommittee/UCM218493.pdf

³ Woodcock Janet, Sharfstein Joshua M., Hamburg Margaret.  Regulatory action on rosiglitazone by the U.S. Food and Drug Administration. New England Journal of Medicine 2010; 363:1489-1491.

What is the Risk of Avandia?

Relative risk is the ratio of a frequency in one group of individuals divided by the frequency in another group of individuals (or controls).  It functions as a multiplier of the probability that an event will occur in one situation, relative to the probability that the same event will occur in another situation.  If the probability, say, of a regular ticket winning a lottery is 0.001% and a premium ticket has a 0.002% probability of winning, the relative “risk” of winning with a premium ticket, compared to a regular ticket, is 0.002%/0.001%=2.  Stated another way, if the relative “risk” of winning with a premium ticket is 2, as compared to a regular ticket, the “risk” of winning with a premium ticket, although still extremely small, is 2 times that with a regular ticket, or (2)(0.001%)=0.002%.¹

Estimates of risk can be of any size.  On the other hand, small estimates of risk are probably methodological noise, no matter what statistical “significance” is calculated for them.  No matter how carefully investigators believe that they have considered all possible errors that could have occurred, actually demonstrating successful consideration is difficult, and questions always remain.  The smaller the estimate of relative risk to more rigorously freedom from errors must be demonstrated.  At some point, it becomes impossible to reduce the errors sufficiently, and the estimate of relative risk becomes meaningless. Viewed the other way around, the larger the relative risk, the less likely it is that factors such as bias or confounding could have overcome a true association.  No matter how large the relative risk, however, cause and effect can never be proven by a relative risk, alone.¹

Calculated confidence intervals can only take into consideration purely random variations in data.  In observational studies, almost invariably, however, nonrandom errors in the data will be more important than random variations.  The most important criteria for evaluating relative risk are a very high calculated risk and a biologic mechanism that is highly plausible.  Most epidemiologists insist upon a relative risk of 3 or 4²–not the puny 1.43 to 1.64 relative risks that Nissen and Wolski claimed³ (and which were inflated,⁴ at that).  Even The New England Journal of Medicine‘s Marcia Angell is quoted as saying that, before accepting a paper to be published, “As a general rule of thumb, we are looking for a relative risk of three or more, particularly if it is biologically implausible or if it’s a brand-new finding.”³  So why did The New England Journal of Medicine accept Nissen and Wolski’s “brand-new finding” that had no biological plausibility and whose relative risks were only half of the Journal‘s minimum requirements?  There is more.  The Food and Drug Administration’s Director of Drug Evaluation, Robert Temple, is quoted as saying, “My basic rule is that if the relative risk isn’t at least three or four, forget it.”  Why, then, did the Food and Drug Administration not do exactly that (forget it) with the Nissen and Wolski publication?

What was the point about brand-new findings, above?  Avoiding the mathematics involved, from Bayes theorem, we know that the probability that a hypothesis is true after an observation (such as Nissen and Wolski’s meta-analysis) is strongly dependent upon the probability that it was true before that observation.  Therefore, meta-analyses and studies that attempt to confirm hypotheses from prior studies are more likely to generate true findings than are studies that produce the hypotheses in the first place.¹

The fraction of risk that can be attributed to a factor (such as rosiglitazone) can be calculated as the attributable risk:  Attributable Risk=(100%)(Relative Risk-1)/(Relative Risk).  If the relative risk is 2, then 50% of the risk is attributable to the factor.  Relative risk must be greater than 2 for more than 50% of the risk to be attributable to the factor and, more likely than not, to have caused the event.¹  That standard, used in litigation, is extremely low, because being wrong in either direction is considered to be equally bad.  Even Nissen and Wolski’s inflated relative risks, however, mustered (at most) a 39% attributable risk–not even rising to the meager “greater-than-50%” standard, let alone being overwhelming evidence against rosiglitazone (Avandia, Avandamet, Avandaryl, GlaxoSmithKline).  Ironically, Peto, whose methodology Nissen and Wolski improperly applied to their meta-analysis, stated, “. . . when relative risk lies between 1 and 2 . . . problems of interpretation may become acute, and it may be extremely difficult to disentangle the various contributions of biased information, confounding of two or more factors, and cause and effect.”⁵

To be sure, the studies on which Nissen and Wolski based their meta-analysis were not purely observational.  The study subjects had been randomly assigned to groups.  However, collection of data about cardiovascular events was not systematic, because they were not prespecified endpoints.  In that sense, a study of the “observations” constitutes an observational study.  For that case, maybe a relative risk of 3 or 4 is too high a standard, but a compromise of at least 2 might be reasonable.

Nissen and Wolski’s meta-analysis was only hypothesis-generating;  the relative risks were inflated by their improper use of statistical techniques; even without systematic biases, the inflated relative risks were still within the ranges expected by chance; however, there were systematic biases against rosiglitazone; even if the relative risks could have been accepted at face value, they would still have been of dubious validity; and the results were not biologically plausible.  As I have already pointed out, in my December 19, 2013 post, “Nissen, Wolski, and How Not to Do a Meta-Analysis,” Nissen and Wolski’s meta-analysis also suffered from multiple subgroup comparisons, seriously further inflating the probability of a statistically “significant” result.¹  In sum, apart from being essentially meaningless, their meta-analyses was otherwise excellent.

© 2013 Myron Shank, M.D., Ph.D.

¹ Nicolich Mark J., Gamble John F.  What is the Minimum Risk that can be Estimated from an Epidemiology Study?  In:  Moldoveanu Anca Maria.  Advanced Topics in Environmental Health and Air Pollution Case Studies InTech, 2011.

² Taubes Gary, Mann Charles C. Epidemiology faces its limits. Science 1995; 269:164-169.

³ Nissen Steven E., Wolski Kathy.  Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes.  The New England Journal of Medicine 2007; 356:2457-2471.

⁴ Diamond George A., Bax Leon, Kaul Sanjay.  Uncertain effects of rosiglitazone on the risk for myocardial infarction and cardiovascular death.  Annals of Internal Medicine 2007; 147:578-581.

⁵ Doll Richard, Peto Richard.  The Causes of Cancer, Oxford-New York: Oxford University Press, 1981, p. 1219.  As quoted in Nicolich Mark J., Gamble John F.  What is the Minimum Risk that can be Estimated from an Epidemiology Study?  In:  Moldoveanu Anca Maria. Advanced Topics in Environmental Health and Air Pollution Case Studies InTech, 2011.

A Little Background on the Big Avandia Controversy

It should be understood that, in 2006, the Food and Drug Administration (FDA) had the results of a meta-analysis, performed by GlaxoSmithKline, which suggested possible increases in cardiovascular risks with rosiglitazone (Avandia, Avandamet, Avandaryl; GlaxoSmithKline),² a year before Nissen and Wolski’s controversial article.¹  In fact, the Food and Drug Administration (FDA) had requested patient-level data for these studies and was already conducting its own meta-analysis.²  The Food and Drug Administration’s meta-analysis included studies that were different from those used by Nissen and Wolski² and had occurred independently of Nissen and Wolski.

Most of the component studies in the Food and Drug Administration’s analysis were of less than six months duration–not very long for cardiovascular (“heart attack,” anginal chest pain, stroke, and sudden death) outcomes.  As with the group of studies included by Nissen and Wolski, none had been designed to evaluate cardiovascular risk, so they had lacked plans for studying cardiovascular events.  Instead, adverse events that were captured in case report forms had been classified using often vague terms (such as “chest pain,” without any laboratory or electrocardiogram (EKG) evidence that the heart was the cause).²  Without adjudication (tracking down and examining the appropriate original data), it is impossible to know what these reports really meant.  As noted in my December 18, 2013 post, “Steven Nissen and The Not-So-Great Avandia Controversy,” Nissen and Wolski did not do this; however, the Food and Drug Administration did.  When compared with placebo (an inactive lookalike), Avandia showed possible increases both in ischemic heart disease (“heart attacks” and anginal chest pain) and in ischemic heart disease plus strokes, but not when compared with active drugs.²  There was a possible increase in ischemic heart disease, but not ischemic heart disease plus strokes, in the combination of studies comparing Avandia either to placebos or to active drugs.²

So long as its results were compatible with his own, Dr. Nissen does not seem to have had any complaints with the Food and Drug Administration’s analyses.  As we will see in future posts, however, that changed, once its results no longer supported his agenda.  In spite of the glaring shortcomings in Nissen and Wolski’s meta-analyses,^1,3 as compared both to another meta-analysis of the same data,⁴ published a mere five months after their first meta-analysis and to those of the Food and Drug Administration,² Dr. Nissen has stubbornly clung to his conclusions.

Dr. Nissen has also viciously attacked the Food and Drug Administration for continuing with its planned evaluations of Avandia.  Why?  If it was only because he knew that his own conclusions would not be confirmed, why does he not seem to have ever lashed out at Diamond and others?⁴  The fact that his ongoing vitriol seems to be targeted against the Food and Drug Administration, but apparently not Diamond’s group, gives credence to speculations that Dr. Nissen fully expected Avandia to be his ticket to the top Food and Drug Administration position. Whether or not he ever sought a formal position, Dr. Nissen seems preoccupied with controlling the Food and Drug Administration.  At any rate, evidence will be reviewed that Dr. Nissen’s motives may be suspect on multiple levels.

© 2013 Myron Shank, M.D., Ph.D.

¹ Nissen Steven E., Wolski Kathy.  Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes.  The New England Journal of Medicine 2007; 356:2457-2471.

² Parks Mary H.  Introduction memorandum.  Readjudication of the Rosiglitazone Evaluated for Cardiovascular Outcomes and Regulation of Glycemia in Diabetes Trial (RECORD):  Joint Meeting of the Endocrinologic and Metabolic Drugs Advisory Committee and the Drug Safety and Risk Management Advisory Committee June 5—6, 2013.  http://www.fda.gov/downloads/advisorycommittees/committeesmeetingmaterials/drugs/endocrinologicandmetabolicdrugsadvisorycommittee/ucm354859.pdf

³ Nissen Steven E., Wolski Kathy.  Rosiglitazone revisited:  an updated meta-analysis of risk for myocardial infarction and cardiovascular mortality.  Archives of Internal Medicine 2010; 170:1191-1201.

⁴ Diamond George A., Bax Leon, Kaul Sanjay.  Uncertain effects of rosiglitazone on the risk for myocardial infarction and cardiovascular death.  Annals of Internal Medicine 2007; 147:578-581.

Nissen, Wolski, and How Not to Do Meta-Analyses

A meta-analysis uses statistical techniques to combine multiple “real” studies into a new “almost as if” study.  A meta-analysis is “almost as if” a study had actually been done the same way that the included studies were done.  The purpose of a meta-analysis is to contrast and combine results, in order to create a composite study more powerful than any of the component studies.  This is what Nissen and Wolski performed for their controversial New England Journal of Medicine publication, which suggested that rosiglitazone (Avandia, Avandamet, and Avandaryl; GlaxoSmithKline) was responsible for an increase in myocardial infarctions (“heart attacks”) and deaths due to cardiovascular (heart or blood vessel) causes.¹  A meta-analysis can never be better than a real study that was designed the same way as the meta-analysis, but it can be worse.  There are a number of pitfalls in performing a meta-analysis.

Both of Nissen and Wolski’s meta-analyses^1,3 were seriously flawed.  They based them upon a biased (non-systematic) selection of component studies, they included studies that arguably should not have been included with the others, they made no attempt to validate the data, and they used unsuitable statistical techniques.²  All of these errors were described for their first meta-analysis¹ in my December 18, 2013 post, “Steven Nissen and The Not-So-Great Avandia Controversy.”  In addition, more patients in the control groups dropped out (because of elevated blood glucose levels), resulting in less follow-up for them than for the rosiglitazone groups.⁴  It is obvious that the longer follow-ups for subjects treated with rosiglitazone resulted in a greater time periods in which to experience a cardiovascular event, biasing the results against rosiglitazone.

Is that all that they did wrong?  No.

A problem known as “multiple comparisons” was a serious defect in both of Nissen and Wolski’s meta-analyses.  As far as I can tell, no one else has pointed out this critical error, at least in print. [Note: Since this posting, I have found a possible reference to the problem:  “The initial concern with rosiglitazone arose from observational and case–control epidemiologic studies that generated a legitimate signal of possible cardiovascular harm, but every study had substantial methodologic shortcomings, including multiplicity, which meant that a statistically positive finding might be a false positive result.”⁵ –Myron Shank, M.D., Ph.D., December 21, 2013.]

Assuming that data represents truly random (chance) samplings, statistics can be used to estimate how frequently differences as large or larger than those observed would have occurred in samples that came from the same population.  It is usually assumed that the data represent different populations, rather than coming from the same population, if, on average, random samples from the same population would be as different as, or more different than, those that were actually observed less than one time out of twenty.  Stated more conventionally, if there is less than a 5% probability (P<0.05) of seeing results as different as or more different than what was observed, the difference is usually said to be “statistically significant” (In reality, the choice of probability should depend upon the relative costs of erroneously concluding that the samples come from the same population or that they come from different populations, but this is seldom considered in medicine.).

The more times that you make comparisons, however, the more likely you are to see unusual differences.  If you sample enough times, you will eventually see even differences that occur very rarely.  The probability (P) is for a single comparison, not for multiple comparisons.

So, what did Nissen and Wolski do?

In their original analysis,¹  Nissen and Wolski performed not one but eighteen comparisons. Their “overall” analyses overlapped not only with their analyses of “small trials,” but with the DREAM and ADOPT trials, as well.  Cardiovascular deaths overlapped with myocardial infarctions.  “Combined comparator drugs” overlapped with the individual drugs (metformin, sulfonylurea, insulin, and placebo).

Worse, in their revised meta-analysis,³ Nissen and Wolski performed not one but thirty-four comparisons.  Moreover, cardiovascular deaths still overlapped with myocardial infarctions, short-term and long-term studies overlapped with “overall” analyses, “overall” analyses overlapped with their component analyses, analysis without RECORD overlapped with those with RECORD, and their “Peto” analyses overlapped with their analyses that included studies without any cardiovascular events or deaths.  It is utterly impossible to fully correct for the errors introduced by these interrelated multiple comparisons, but, with their marginally “significant” results, even the most timid attempt would have completely eliminated any appearance of “significant” differences.

Fortunately, there is an easy solution.  If, and only if, their “overall” analysis for all cardiovascular outcomes and deaths were statistically “significant,” Nissen and Wolski were justified in performing sub-analyses. In other words, if they knew that there was a difference somewhere in the data, Nissen and Wolski were entitled to search for it without fear of falsely creating the appearance of a difference.  This is sometimes known as a “protected” statistical test, because the sub-analysis is “protected” by the knowledge that the overall results were unlikely to have occurred by chance.  In their original meta-analysis, Nissen and Wolski could properly have performed their sub-analyses, if a correct analysis of data had appeared to show overall “significant” differences between rosiglitazone and other drugs or placebo.  However, as I discussed in my December 18, 2013 post, “Steven Nissen and The Not-So-Great Avandia Controversy,” that was not the case.  In the absence of a true “overall” difference, comparing rosiglitazone with all non-rosiglitazone treatments, their sub-analyses was invalid.

While someone might attempt to excuse them for this statistical breach, on the basis that Nissen and Wolski (incorrectly) believed that their “overall” results were “significantly” different,¹ no such rationalization is possible for their revised meta-analysis.  In that study, their “overall” results unambiguously showed no difference between treatment with rosiglitazone and treatment without rosiglitazone.³  Since their “overall” results were not “significantly” different, Nissen and Wolski were not entitled to test for the source of the non-existent difference.  They proceeded to do just exactly that, anyway, claiming to find “significant” differences³ that, by definition, did not exist in the data.  Worse, Dr. Nissen has adamantly insisted that these phoney results trump everything else.

The bottom line is that Nissen and Wolski’s own revised meta-analysis failed to show any evidence of a difference between treatments that included rosiglitazone and those that did not.  Period.  All of the hubris in the world cannot overcome that simple fact.

©2013 Myron Shank, M.D., Ph.D.     

¹ Nissen Steven E., Wolski Kathy.  Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes.  The New England Journal of Medicine 2007; 356:2457-2471.

² Diamond George A., Bax Leon, Kaul Sanjay.  Uncertain effects of rosiglitazone on the risk for myocardial infarction and cardiovascular death.  Annals of Internal Medicine 2007; 147:578-581.

³ Nissen Steven E., Wolski Kathy. Rosiglitazone revisited: an updated meta-analysis of risk for myocardial infarction and cardiovascular mortality. Archives of Internal Medicine 2010; 170:1191-1201.

⁴ Home Philip D., Pocock Stuart J., Beck-Nielsen Henning, Gomis Ramón, Hanefeld Markolf, Jones Nigel P., Komajda Michel, McMurray John J.V., for the RECORD Study Group.  Rosiglitazone evaluated for cardiovascular outcomes—an interim analysis.  New England Journal of Medicine 2007; 357:28-38.

⁵ Hiatt William R., Kaul Sanjay, Smith Robert J. The cardiovascular safety of diabetes drugs—insights from the rosiglitazone experience. New England Journal of Medicine 2013; 369:1285-1287.