Yang Q, She H, Gearing M, Colla E, Lee M, Shacka JJ, Mao Z.
Regulation of neuronal survival factor MEF2D by chaperone-mediated autophagy.
Science. 2009 Jan 2;323(5910):124-7.
PubMed.
Don’t Jeopardize New Therapies With Sham Surgery Control—Placebo Responses May Be Part of Therapies
I was the patient representative on the FDA advisory panel that reviewed deep brain stimulation (DBS) in March of 2000, and later I participated in the Medicare National Coverage decision for DBS on behalf of the requester (not Medtronics but an individual person with Parkinson's). From these engagements, I recall this treatment was shown to be very effective (upwards of 85 percent have 50 percent improvement in motor symptoms). Such dramatic and lasting improvements would need to be expected to offer a treatment that makes it worthwhile to take the risk of brain surgery. After a delay of more than four years from the initial advisory group, DBS has been available to patients, as a near breakthrough option once first-line treatment fails. Indeed, it is the only major new therapy for PD in the 40 years since levodopa was discovered. Now the recent study published in JAMA continues to show efficacy and also shows that its adverse side effects for important functions like cognition are greater for DBS than with standard drug therapy. The DBS experience can be instructive for other surgical treatments for PD.
The question I want to raise regards the Ceregene 120 study, a gene therapy application of the nerve growth factor neurturin, NTN. The Phase 2 study "failed to meet primary endpoints" in comparison to a sham surgery placebo control. Similar to experiences with other surgically installed treatments, such as GDNF infusion pump and implantation of spheramine, a cell-based therapy using retinal dopaminergic cells, this latest placebo-controlled trial did not replicate the gains from the Phase 1 open-label study. The results of all of these studies are clouded from methodological issues such as differences in dosing between Phase 1 and Phase 2, dislodgement of pump connections, and differences in the use of other PD medications during the studies that may have affected the results. Even so, the fact remains that some study participants have experienced dramatic improvements lasting as long as six years and counting, and some have reduced their PD medications to near zero, being almost symptom-free after decades of increasing disability. A brain autopsy of one GDNF patient showed nerve growth in the side of the brain in which the treatment was administered during the trial. For all of these treatments, data point to improvements well beyond reasonable estimates of placebo effects.
Clearly placebo effects are very strong. Research on placebo response for a range of medical conditions including PD attributes these real physiological effects to expectations of benefit and conditioning established in the social context of administering a treatment. The greater the risk and notoriety of the intervention, and the more certain and authoritative the source, a greater placebo effect is produced. Maximum placebo effects, as would be expected, are found from brain surgery as well as from the safest form of sham brain surgery, where the brain is not penetrated but the patient goes through the same process including lengthy anesthesia. DBS patients report vast improvements in symptoms even before the stimulators are turned on. Clinical brain researchers (including more than 90 percent of the Parkinson's Study Group) agree that sham brain surgery is necessary to prove that improvements are attributable to the treatment beyond the placebo. On the other hand, an online survey of activist PD patients found that only 37 percent would participate in a sham surgery study. Closing this gap raises practical as well as ethical issues.
DBS was approved without sham surgery as a placebo control. So why aren't DBS's gains in motor scores attributed in part to a learned placebo response? Shouldn't the placebo effects that last multiple years be counted as part of the treatment, as they effectively are with DBS, and not written off as bias? The recent JAMA publication improved the evidence of efficacy for DBS by randomization to best medical treatment versus surgery. Why isn't random assignment to best medical practice a sufficient comparison for other surgical interventions?
Sham surgery is not a sugar pill; it is a powerful intervention, although you would probably be charged with fraud if you tried to sell it. Placebo studies on the experience of pain in fact demonstrate that the "bias" from patient hopes and expectations, a central element of all healing, is opposite of what has been assumed by science in experimental settings. That is, treatment effects are reduced and placebo effects are increased. That is so because random assignment dilutes positive effect of patients’ expectation that they will improve from the ongoing uncertainty about whether they are on the real thing, and conversely it elevates the placebo group’s expectation that they may be on the real thing. This biases the results toward type II errors (false negative), which are more important to patients with serious illness than are type I errors (false positive) that are the target of statistical models. The pain studies suggest that the placebo mechanism may be necessary to trigger the therapeutic effects of treatment. Elaborate deception to control this effect could be undermining its evaluation.
Sure, we need to control bias. But variability and bias can come from many sources, including, importantly the selection of participants and the variability of raters on subjective scales such as UPDRS. For example, are all study participants diagnosed correctly? And do they represent homogeneous types of patients? Depression medication trials, which also fail at high rates, have taught us that clearer distinction between treatment and placebo results from higher-quality central rating of subjective measurement scales. Multiple ratings of key measures reduce noise in data when averaged. Patients who have participated in PD clinical trials know that UPDRS "off" may describe different behavior on different days, and are not totally determined by the time since the last dose. Better understanding of these factors from the patient perspective is necessary to control this source of variability in the data.
Alternatively, where the sources of variability are unbiased, the problem can be fixed by increasing sample size to account for random fluctuations in the calculations of confidence of the result. This not only costs more, but it also may bump up against practical limitations including recruitment and FDA statisticians.
New Directions for the Twenty-first Century
As science progresses, we need to re-examine our assumptions about the standards for evidence in the assessment of safety and effectiveness. The gold standard of the randomized, prospective, double-blind placebo-controlled study cannot be applied as a one-size-fits-all to conditions on the cutting edge of medical science.
Medical miracles of the twentieth century mostly pertain to acute conditions, where linear assumptions of statistical models for hypothesis testing more closely approximate the relatively short-term interventions. As we deal with longer-term degenerative processes involving dynamic interaction and feedback to our conscious brain processes, assumptions from the experimental model become questionable. This is true even where all orthodoxies of statistics are followed and statistical significance is achieved.
Recent FDA law offers greater flexibility to align methods to the parameters of the specific case. Such alternative methods need to be qualified and used. Examples include Bayesian statistics for application to dose-finding tasks, or mathematical models of disease progression as historical controls. Crossover designs can detect differences in symptomatic benefits, and delayed start designs have shown promise to detect neuroprotection.
FDA law requires well-controlled randomized studies, not placebos. New policies put more emphasis on life cycle monitoring of treatments in real practice settings, and provide reimbursement coverage for access to new treatments while evidence of long-term safety and efficacy are established with greater certainty. Following patients more closely for a number of years to see the lasting effects can establish whether the treatment effects are purely placebo, at the same time that long-term safety is tracked.
Continuing failure of studies based on faulty assumptions about human behavior is not a viable option. A better understanding of placebo responses in the design of clinical trials points to new approaches in collaboration with patient advocates and communications to FDA.
Comments
Parkinson Pipeline Project, C-Path Inst.
Don’t Jeopardize New Therapies With Sham Surgery Control—Placebo Responses May Be Part of Therapies
I was the patient representative on the FDA advisory panel that reviewed deep brain stimulation (DBS) in March of 2000, and later I participated in the Medicare National Coverage decision for DBS on behalf of the requester (not Medtronics but an individual person with Parkinson's). From these engagements, I recall this treatment was shown to be very effective (upwards of 85 percent have 50 percent improvement in motor symptoms). Such dramatic and lasting improvements would need to be expected to offer a treatment that makes it worthwhile to take the risk of brain surgery. After a delay of more than four years from the initial advisory group, DBS has been available to patients, as a near breakthrough option once first-line treatment fails. Indeed, it is the only major new therapy for PD in the 40 years since levodopa was discovered. Now the recent study published in JAMA continues to show efficacy and also shows that its adverse side effects for important functions like cognition are greater for DBS than with standard drug therapy. The DBS experience can be instructive for other surgical treatments for PD.
The question I want to raise regards the Ceregene 120 study, a gene therapy application of the nerve growth factor neurturin, NTN. The Phase 2 study "failed to meet primary endpoints" in comparison to a sham surgery placebo control. Similar to experiences with other surgically installed treatments, such as GDNF infusion pump and implantation of spheramine, a cell-based therapy using retinal dopaminergic cells, this latest placebo-controlled trial did not replicate the gains from the Phase 1 open-label study. The results of all of these studies are clouded from methodological issues such as differences in dosing between Phase 1 and Phase 2, dislodgement of pump connections, and differences in the use of other PD medications during the studies that may have affected the results. Even so, the fact remains that some study participants have experienced dramatic improvements lasting as long as six years and counting, and some have reduced their PD medications to near zero, being almost symptom-free after decades of increasing disability. A brain autopsy of one GDNF patient showed nerve growth in the side of the brain in which the treatment was administered during the trial. For all of these treatments, data point to improvements well beyond reasonable estimates of placebo effects.
Clearly placebo effects are very strong. Research on placebo response for a range of medical conditions including PD attributes these real physiological effects to expectations of benefit and conditioning established in the social context of administering a treatment. The greater the risk and notoriety of the intervention, and the more certain and authoritative the source, a greater placebo effect is produced. Maximum placebo effects, as would be expected, are found from brain surgery as well as from the safest form of sham brain surgery, where the brain is not penetrated but the patient goes through the same process including lengthy anesthesia. DBS patients report vast improvements in symptoms even before the stimulators are turned on. Clinical brain researchers (including more than 90 percent of the Parkinson's Study Group) agree that sham brain surgery is necessary to prove that improvements are attributable to the treatment beyond the placebo. On the other hand, an online survey of activist PD patients found that only 37 percent would participate in a sham surgery study. Closing this gap raises practical as well as ethical issues.
DBS was approved without sham surgery as a placebo control. So why aren't DBS's gains in motor scores attributed in part to a learned placebo response? Shouldn't the placebo effects that last multiple years be counted as part of the treatment, as they effectively are with DBS, and not written off as bias? The recent JAMA publication improved the evidence of efficacy for DBS by randomization to best medical treatment versus surgery. Why isn't random assignment to best medical practice a sufficient comparison for other surgical interventions?
Sham surgery is not a sugar pill; it is a powerful intervention, although you would probably be charged with fraud if you tried to sell it. Placebo studies on the experience of pain in fact demonstrate that the "bias" from patient hopes and expectations, a central element of all healing, is opposite of what has been assumed by science in experimental settings. That is, treatment effects are reduced and placebo effects are increased. That is so because random assignment dilutes positive effect of patients’ expectation that they will improve from the ongoing uncertainty about whether they are on the real thing, and conversely it elevates the placebo group’s expectation that they may be on the real thing. This biases the results toward type II errors (false negative), which are more important to patients with serious illness than are type I errors (false positive) that are the target of statistical models. The pain studies suggest that the placebo mechanism may be necessary to trigger the therapeutic effects of treatment. Elaborate deception to control this effect could be undermining its evaluation.
Sure, we need to control bias. But variability and bias can come from many sources, including, importantly the selection of participants and the variability of raters on subjective scales such as UPDRS. For example, are all study participants diagnosed correctly? And do they represent homogeneous types of patients? Depression medication trials, which also fail at high rates, have taught us that clearer distinction between treatment and placebo results from higher-quality central rating of subjective measurement scales. Multiple ratings of key measures reduce noise in data when averaged. Patients who have participated in PD clinical trials know that UPDRS "off" may describe different behavior on different days, and are not totally determined by the time since the last dose. Better understanding of these factors from the patient perspective is necessary to control this source of variability in the data.
Alternatively, where the sources of variability are unbiased, the problem can be fixed by increasing sample size to account for random fluctuations in the calculations of confidence of the result. This not only costs more, but it also may bump up against practical limitations including recruitment and FDA statisticians.
New Directions for the Twenty-first Century
As science progresses, we need to re-examine our assumptions about the standards for evidence in the assessment of safety and effectiveness. The gold standard of the randomized, prospective, double-blind placebo-controlled study cannot be applied as a one-size-fits-all to conditions on the cutting edge of medical science.
Medical miracles of the twentieth century mostly pertain to acute conditions, where linear assumptions of statistical models for hypothesis testing more closely approximate the relatively short-term interventions. As we deal with longer-term degenerative processes involving dynamic interaction and feedback to our conscious brain processes, assumptions from the experimental model become questionable. This is true even where all orthodoxies of statistics are followed and statistical significance is achieved.
Recent FDA law offers greater flexibility to align methods to the parameters of the specific case. Such alternative methods need to be qualified and used. Examples include Bayesian statistics for application to dose-finding tasks, or mathematical models of disease progression as historical controls. Crossover designs can detect differences in symptomatic benefits, and delayed start designs have shown promise to detect neuroprotection.
FDA law requires well-controlled randomized studies, not placebos. New policies put more emphasis on life cycle monitoring of treatments in real practice settings, and provide reimbursement coverage for access to new treatments while evidence of long-term safety and efficacy are established with greater certainty. Following patients more closely for a number of years to see the lasting effects can establish whether the treatment effects are purely placebo, at the same time that long-term safety is tracked.
Continuing failure of studies based on faulty assumptions about human behavior is not a viable option. A better understanding of placebo responses in the design of clinical trials points to new approaches in collaboration with patient advocates and communications to FDA.
Perry D. Cohen directs the Parkinson Pipeline Project.
View all comments by Perry CohenMake a Comment
To make a comment you must login or register.