http://www.bmj.com/content/359/bmj.j4528?utm_medium=email&utm_campaign_name=201710278&utm_source=etoc_daily BMJ 2017; 359 doi: https://doi.org/10.1136/bmj.j4528 (Published 04 October 2017)
Editorial
You don’t
need to know, according to our broken regulatory system
When in the
lifecycle of a cancer drug should an improvement in survival or quality of life
be demonstrated? Some people argue that these benefits should be clearly
evident before marketing. Others, me included, believe that for some
indications, including terminal cancers with few treatment options, a drug
might receive provisional approval based on surrogate outcomes (such as tumour
shrinkage or progression-free survival), with overall survival or quality of
life assessed after market authorisation. Although there is no consensus, the
one answer that seems absolutely unjustifiable is never. And yet, this is often
what happens, according to two recent studies.
The first found that between 2008 and 2012 the
US Food and Drug
Administration approved most uses of cancer drugs without evidence of survival
or improved quality of life (67%, 36/54).1 Among the
36 such approvals, only five (14%) uses were shown later to improve survival
compared with existing treatments or placebo after a median of 4.4 years on the
market.
The linked paper by Davis and colleagues (doi:10.1136/bmj.j4530) extends these findings.2In their study of
cancer drugs approved by the European Medicines Agency between 2009 and 2013,
57% (39/68) had no supporting evidence of better survival or quality of life
when they entered the market. After a median of 5.9 years on the market, just
six of these 39 (15%) agents had been shown to improve survival or quality of
life.
Minimal benefit
Three
further facts help characterise the current regulatory climate. Firstly, when
drugs do offer survival advantages, the gains are often marginal. Fojo and
colleagues found that the median improvement in survival among patients treated
with 71 drugs for solid tumours was just 2.1 months.3 Davis and colleagues
agree. Of the 23 drugs that improved survival, 11 (48%) failed to meet the
modest definition of “clinically meaningful benefit” set by the European
Society of Medical Oncology.
All
three aforementioned analyses123 consider only measured
improvements in survival and not mathematically derived or modelled estimates.
This is for good reason. Modelled estimates make assumptions to predict
survival benefits that might occur during longer follow-up or adjusting for
differences in post-study treatment between groups. Modelled estimates are
uncertain and seem to be consistently larger than measured gains,4 raising question about
their fidelity.
Secondly,
the small benefits of cancer drugs typically occur in trials conducted in
unrepresentative patient populations—patients who are younger and with less
comorbidity than average clinical populations.5 When a marginal drug
advantage is applied to a real world population, a small benefit may vanish
entirely because of the fine balance between risks and benefits typical of
these agents.5
Finally,
many of the surrogate outcomes used for drug approval are poorly correlated
with survival.6 For others, the
strength of the correlation is untested.7 This is true for the
FDA’s regular approval pathway as well as the accelerated approval route.6 Notably, regular
approvals are not usually coupled to postmarketing requirements for further
trials to confirm effectiveness and safety. This means that the surrogate
outcome, often unvalidated, may be all we ever have.
Better
patient value
Taken
together, these facts paint a sobering picture. Although we are approving
cancer drugs at a rapid pace, few come to market with good evidence that they
improve patient centred outcomes. If they do, they often offer marginal
benefits that may be lost in the heterogeneous patients of the real world. Most
approvals of cancer drugs are based on flimsy or untested surrogate endpoints,
and postmarketing studies rarely validate the efficacy and safety of these
drugs on patient centred endpoints. Add to this that the average cancer drug
costs in excess of $100 000 (£75 000; €85 000) per year of treatment, and the
conclusion seems that the regulatory system is broken.7
In
the US, this broken system means huge expenditures on cancer drugs with certain
toxicity but uncertain benefit. The US Medicare programme is legally required
to pay for any drug approved by the FDA without negotiation on price.7
In
Europe, agencies such as the National Institute for Health and Care Excellence
(NICE) exclude from reimbursement drugs that provide only marginal or uncertain
benefits at high cost.8 Their decisions are
continually subjected to political scrutiny and public criticism. However, it
is only because regulators are lax that payers have had to wield the stick.
What can be done? The default
path to market for all cancer drugs should include rigorous testing against the
best standard of care in randomised trials powered to rule in or rule out a
clinically meaningful difference in patient centred outcomes in a
representative population. The use of uncontrolled study designs or surrogate
endpoints should be the exception not the rule. When surrogates are used,
postmarketing studies with clinically meaningful and patient centred outcomes
must be started, completed, and published. Patient level data should be shared.
Health technology assessment programmes should reject modelled measurements of
survival, which may unintentionally incentivise the industry not to conduct
trials that evaluate survival directly and rely instead on modelling.
The expense and toxicity of
cancer drugs means we have an obligation to expose patients to treatment only
when they can reasonably expect an improvement in survival or quality of life.
The study by Davis and colleagues suggests we may be falling far short of this
important benchmark.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
http://www.bmj.com/content/359/bmj.j4543?utm_medium=email&utm_campaign_name=201710278&utm_source=etoc_daily
Feature Drug Regulation
Cancer drugs: high price, uncertain value
BMJ 2017; 359 doi: https://doi.org/10.1136/bmj.j4543 (Published 04 October 2017)Cite this as: BMJ 2017;359:j4543
A study published
in The BMJ this week shows how most new cancer drugs are failing to deliver any
clinically meaningful benefit. It’s time for Europe to raise the evidence bar
before market approval, finds
Deborah Cohen
dcohen@bmj,com
Most cancer drugs recently entering the European
market do so
without clear evidence of extending or improving quality of life, new research
published in The BMJ has found.1
The findings
raise serious questions about why the current regulatory environment supports
the approval of cancer drugs that may leave patients at risk of experiencing
toxicity and reduced quality of life without deriving meaningful benefit.
Out of the 68
cancer drug indications approved by the European Medicines Agency during
2009-13, 57% (39) entered the market without evidence of a survival or quality
of life benefit. Even when drugs did show survival gains over available
treatment options, most of these were not clinically meaningful, researchers found.
Many of the drugs
were approved on the basis of surrogate endpoints despite evidence that these
are not a reliable indicator of overall survival or quality of life for most
cancer treatments.
“When expensive
drugs that lack clinically meaningful benefits are approved and reimbursed
within publicly funded healthcare systems, individual patients may be harmed,
important resources wasted, and the delivery of equitable and affordable care
is undermined,” the researchers say.
The study comes
at a time when European governments are starting to seriously challenge the
high cost of drugs. While it’s hard to know how much healthcare systems are
paying for cancer drugs because prices are often negotiated behind closed
doors, the total amount spent on cancer care is growing, partly because of the
cost of drugs.
Inadequate
evidence
The research found that the
EMA is basing many approval decisions on uncontrolled study designs or
surrogate endpoints, which don’t always translate into outcomes which make a
difference to patients.
Some
of the cancer drugs were given “conditional marketing authorisations,” on the
understanding that postmarketing studies would assess overall survival or
quality of life. If the drugs are subsequently found to be clinically
ineffective or unsafe, then the EMA can withdraw them. The study in The BMJ identifies
10 drugs
approved under these fast track arrangements, but after four years of market
entry none of them had good evidence that they either extended or improved life
for patients.
Asked
to comment on the research by The
BMJ, an EMA spokesperson stated that is not agency practice to
comment on research it has not seen. This is a topic that it has discussed
widely and it welcomes taking part in further debate on the evidence
underpinning cancer medicines.
The fact that so many of the
new drugs on the market lack good evidence that they improve patient outcomes
puts governments in a difficult position when it comes to deciding which
treatments to fund.
In 2016, European health
ministers issued a statement saying that new medical products “pose challenges
to individual patients and public health systems in particular regarding their
added value.” This, they said, affects patient access, affordability, and the
financial sustainability of health systems.
As an example of the
pressures put on health systems, a recent Bristol-Myers Squibb funded analysis
compared licensed drugs for six different cancers across Europe and Canada and
concluded that reimbursement decisions seem inconsistent. In an accompanying
press release, the authors of the analysis stated: “There are potentially
200 000 patients in 12 countries who by licence should have access to drugs but
are not getting them because of the reimbursement decision.”
Their
underlying premise was that the EMA grants licences to “safe, effective cancer
treatments where access to the drug can improve and prolong life” and these
drugs should be paid for.2
In
the eyes of industry the problem is with national reimbursement, yet the
overwhelming picture now, from not only The
BMJ study1 but from
studies published in Lancet
Oncologyand elsewhere, is that cancer medicines are being licensed
that do not deliver clinically meaningful benefit.
The BMJ has found
methodological problems with trials that EMA has
either failed to identify or overlooked (box 1). This includes the trials’
design, conduct, analysis, and reporting.
Box 1: EMA’S
questionable practices
Lenience on
trial design
EMA’s approval of mifamurtide in 2009 for non-metastatic
osteosarcoma was on the basis of a pivotal trial that was not designed to
evaluate the drug’s clinical efficacy. Instead, the factorial design trial was
aimed at independently evaluating the effect of mifamurtide and another agent,
ifosfamide. In 2007, the FDA’s oncologic drugs advisory committee raised
serious concerns about the study design and choice of comparators and concluded
that mifamurtide failed to show substantial evidence of efficacy
Pomalidomide was approved by the EMA in 2013 for relapsed or
refractory multiple myeloma. The trial supporting this indication compared one
group of patients who were randomly allocated to receive pomalidomide in
combination with low dose dexamethasone to another group receiving high dose
dexamethasone. The company was repeatedly alerted about the unsuitability of
the comparator choice for regulatory decision making by the FDA and NICE
Failure to
follow its recommendations
The main study supporting the EMA’s marketing authorisation
of
S-1 (tegafur/gimeracil/oteracil) in combination with cisplatin for the
treatment of advanced gastric cancer was designed and conducted to show its
superiority over 5-fluorouracil in combination with cisplatin. When the trial
did not meet its objective, and after it was completed, the sponsor company
considered it appropriate to change the primary objective from superiority to
non-inferiority. EMA’s guidance from 1999 admitted that “there is ample room
for bias” if the non-inferiority margin is chosen after the data have been
inspected
Tolerance for
questionable analytical practices
In 2011, EMA approved bevacizumab as a first line treatment
for
advanced ovarian cancer. Two pivotal studies supported this extension of
bevacizumab’s previously approved indication. In one of these studies, the
primary endpoint was initially specified as overall survival but was changed to
progression-free survival during the trial, jeopardising a fundamental
principle in trial design.
Vinflunine was approved by the EMA in 2009 as a second line
treatment for metastatic transitional cell carcinoma of the urothelial tract.
In its pivotal trial, vinflunine was associated with an overall survival
benefit over best supportive care. However, this effect was observed only in
the those treated per protocol and not in the intention-to-treat population
Trastuzumab received marketing authorisation
from the EMA in
2011 for locally advanced breast cancer. The main study supporting this
approval measured overall survival as a secondary endpoint. Although the
predefined analysis did not show a statistically significant survival benefit,
EMA concluded that “the strongest evidence of benefit was provided by overall
survival results” on the basis of an exploratory analysis that achieved
significance after data were excluded from one of the participating centres
“for which issues were raised.” In the five year follow-up of the trial,
investigators concluded that the difference between groups in overall survival
did not reach significance, probably because of crossover to adjuvant
trastuzumab in some patients initially randomised to chemotherapy alone
Panitumumab was approved by the EMA in 2011 as
a second line
treatment for colorectal cancer on the basis of a randomised controlled trial
with the coprimary endpoints progression-free survival and overall survival.
According to the predefined statistical analysis plan, progression-free
survival and overall survival were analysed using a P value of 0.01 and 0.04,
respectively.3 At the time of
approval, primary analysis showed a borderline statistically significant
benefit in progression-free survival, which the EMA did not consider robust.
There was no statistically detectable overall survival benefit. However, a
later publication in the postmarketing period reported that the final analysis
of this trial showed significant improvement in progression-free survival.4 Yet the P value for
this analysis was 0.023, which did not meet the investigators’ prespecified
threshold.
Reporting by Courtney Davis, Department of Global Health and
Social Medicine, King’s College, London, and Huseyin Naci, LSE Health, London
Such flawed clinical trials
can lead to bias and further difficulties in identifying the true
effectiveness. Unless there’s thorough scrutiny of this regulatory evidence
after approval, governments may make poor decisions about how to prioritise
health budgets.
Unrealistic
expectations
Perhaps most importantly,
however, the fact the drugs have been given the imprimatur of regulatory
approval may cause patients and doctors to have unrealistic expectations about
their benefits and harms.
According to Richard
Sullivan, professor of cancer and global health at King’s College London and
director of the Institute of Cancer Policy, doctors cannot be expected to be
gatekeepers. In many cases across Europe new cancer medicines with low
clinically meaningful benefit continue to be prescribed.
“They may inappropriately
script cancer drugs because of patient and family pressure; a lack of
understanding of how new complex therapies work; or because of the culture of
medical oncology in the absence of multidisciplinary decision making,” he says.
“If patients are not offered alternative modalities, including palliative care,
in end-of-life settings then the risk of inappropriate or futile treatment with
chemotherapy and immunotherapy increases.”
Sullivan says the processes
that allow a drug to be funded in national health services across Europe vary
in their robustness and due diligence around judging clinical evidence. Some
health technology assessment bodies view themselves as secondary gatekeepers to
stop use of drugs that the EMA has licensed without evidence of benefit (box
2).
Box 2: EMA under
scrutiny
Some
aspects of the EMA’s regulatory process are coming under scrutiny—not least the
scientific advice offered companies seeking approval for their drugs. According
to Guido Rasi, EMA’s executive director, early dialogue with medicine developers
allows the agency to give scientific advice and help with protocols to “provide
methodological direction and discourage the production of irrelevant or
substandard data.”5
Currently, the scientific
advice EMA gives is not publicly available, preventing its assessment. Some
advocacy groups have reported that attempts to access such information have
been thwarted by commercial confidentiality.
People
both currently and formerly involved with EMA have told The BMJ that manufacturers
see presubmission processes as a way to lobby the agency, repeatedly asking the
same question until they get the answer they want, and this may affect various
aspects of trial design, conduct and analysis.
In July this year Emily O'
Reilly, the European Ombudsman, launched a “strategic inquiry” into EMA’s
processes.
Although she recognised that
“these activities help the development and availability of high-quality,
effective and acceptably safe medicines,” such “activities may pose some
risks.”
She
noted that the EMA sees presubmission meetings as a way to “enable medicine
developers to establish contact with the agency staff who will be involved with
the application.”5
The case of cancer drugs also
raises question about inconsistency between funding and licensing decisions in
Europe. One suggestion has been for the EMA to work alongside the organisations
carrying out health technology assessments, which countries use to help decide
whether to pay for a treatment.
On paper, joining up the EMA
with health technology assessment might seem a quicker way of getting drugs to
patients. But concerns are being raised that EMA standards might actually erode
those applied by some health technology bodies. More importantly, perhaps, is
that decisions about pricing and reimbursement should also be related to the
gross domestic product of countries, which differs greatly across the EU.
Differential pricing would not be possible if assessment was linked with EMA
approval.
Furthermore,
members of some health technology bodies say they are currently a barrier
against poor regulatory decisions and worry that EMA’s “regulatory
capture”—whereby 89% of the agency’s budget comes from the drug industry
fees—may come to thwart their independence.6
For example, the EMA approved
vinflunine as a second line treatment for metastatic transitional cell
carcinoma of the urothelial tract in 2009 (box 1) on the basis of a potentially
biased analysis. But the UK National Institute for Health and Care Excellence
(NICE) was less convinced, implying the evidence used for regulatory approval
did not show the drug to be effective, and didn’t recommend it.
Although
the exact reasons for rejection aren’t always clear, an assessment body
analysis obtained by The
BMJ shows that health technology assessments in most European
countries have also taken a less favourable view of vinflunine than the EMA.
For
patients, approval of such drugs may lead to unrealistic expectations, fuelled
by patient organisations . In response to NICE’s decision, Action on Bladder
Cancer, a charity supporting patients and promoting research, wrote to the
agency to complain that: “Patients with metastatic bladder cancer are
disadvantaged by the lack of a second line treatment option. Study 302 is the
first trial to show a survival benefit and we feel that vinflunine should be
available for this relatively small group of patients.”7
Uncertainty
is compounded by unproved drugs being used as comparators. Despite the
questions around vinflunine , for example, it is now being used as a comparator
in trials for new drugs. On its website, the patient charity Fight Bladder
Cancer has highlighted an ongoing study of a drug called PDL3280A for patients
with advanced or metastatic bladder cancer.8 This, they say,
compares chemotherapy with either paclitaxel or vinflunine. But regulatory
sanctioning of a comparator that lacks robust evidence of efficacy, means the
cycle of weak evidence and uncertainty continues.
No one wants
to say no to a cancer drug
Vinflunine
isn’t an isolated example of questionable decision making. In 2011, the EMA
licensed panitumumab in combination with other drugs as a second line treatment
for colorectal cancer. This was despite the agency questioning whether the
primary analysis showing a borderline statistically significant benefit in
progression-free survival was robust (box 1). Indeed, the EMA initially
rejected the drug but later reversed its opinion.9 In the words of one EMA
adviser who spoke to The
BMJ, however, “no one wants to say no to a cancer drug.” When NICE
invited Amgen, panitumumab’s manufacturer, to submit evidence for approval to
use on the NHS, the company declined to do so. They intimated that there wasn’t
sufficient evidence to determine the cost effectiveness. Again NICE didn’t
recommend it.10
Most
European funding bodies have also turned down panitumumab for second line
treatment for colorectal cancer. However, the published study does not reflect
the questions over the statistical analyses, concluding that panitumumab
“significantly improved” progression-free survival and there was “a trend
toward” improved overall survival.4
Subsequent hype and
misleading reporting can put pressure on governmental bodies to reimburse a drug.
Health technology assessors are in an invidious position—if they refuse to
reimburse and a drug later turns out to be an important therapeutic advance,
then patients have lost out because of the delay. If they reimburse and the
drugs later turn out to be ineffective, have no clinically meaningful effects,
or not to be cost effective then patients may have been unnecessarily subjected
to toxic drugs and scarce healthcare resources have been wasted.
The combination of this hype
coupled with underlying concerns about the quality of trials can cause
confusion. In England, pomalidomide, an immunomodulatory drug for refractory
multiple myeloma wasn’t available, then was, then wasn’t, then was again.
The drug was approved by the
drug regulators both in Europe and the US despite questions over trial design
(box 1).
Correspondence
between the FDA and Celgene, the trial sponsor, shows the US agency cautioning
the company that its choice of comparator was unsuitable for regulatory
decision making.1112 The FDA ultimately
ignored its own advice and approved the drug regardless. The EMA approved it
several years later.
When
NICE rejected the drug for use in the NHS because of the poor comparator,
Celgene claimed that it was chosen only after consulting with the regulators.
Pomalidomide subsequently went onto England’s Cancer Drugs Fund, an extra
source of funding, only for budgetary constraints eventually leading to it
being removed in 2015. It was one of 17 drugs for 23 different indications to
be included in the cull.13
The delisting prompted both
companies and charities to call for all interested parties to work together to
find a better solution for patients, putting pressure on NICE. Noticeably,
however, this did not necessarily include better evidence generation and better
oversight from the EMA.
“This
has been a unilateral decision by NHS England. What is missing is a willingness
for all stakeholders to take part in collaborative discussion and work
together,” Wim Souverijns, general manager at Celgene UK and Ireland, said.
“There is a role for companies to put pressure on stakeholders, including NICE
and NHS England, to point out the implications of these changes.”13
This pressure may have worked
for patient access and cost purposes. In 2016—after the company offered a
confidential discount—NICE approved the drug because it then became cost
effective. But that does not mean the evidence of efficacy is robust; other
European countries continue to deem it not worth paying for.
Regulator fit
for purpose?
The inability of EMA to
uphold its current policies on drug approval has implications for patients and
budgets, and the scientific advice the agency gives is coming under increasing
scrutiny (box 2).
The
example of bevacizumab (Avastin) is a cautionary tale about how well the EMA
can monitor, evaluate, and learn about products even after they are on the
market. Yet the agency wants to push ahead with a programme allowing quicker
access to drugs with immature data.14
In
June 2011, the FDA announced it had revoked bevacizumab’s indication for
metastatic breast cancer because it “has not been shown to provide a benefit,
in terms of delay in the growth of tumors, that would justify its serious and
potentially life-threatening risks. Nor is there evidence that [it] will either
help women with breast cancer live longer or improve their quality of life.”15
But
the drug is still licensed in Europe for metastatic breast cancer; the EMA
withdrew the licence for only some uses. It said this was because the available
data on use in combination with paclitaxel have “convincingly shown [the drug]
to prolong progression-free survival of breast cancer patients without a
negative effect on the overall survival.”16
Again
the EMA’s decision caused challenges for funders. Bevacizumab was rejected by
NICE and was one of the most requested drugs under the Cancer Drugs Fund.17
It
was another of the drugs delisted in 2015, leading to disappointment of patient
groups. “People with incurable breast cancer can only watch from the sidelines
as life-extending treatments are debated again and again and vital options
disappear,” the charity Breast Cancer Care said in a statement.18 But patients will
continue to have their hopes dashed if regulators approve drugs using designs
that are not methodologically rigorous.
