I am going to discuss the failings of the PREDIMED study, real and imagined. I will do this in a Question & Answer style. Several questions are inspired by theheart.org and NEJM comments and even statements by the low fat "gurus", Essselstyn, Ornish, McDougall or Pritikin. It is no surprise they are up in arms after the study has been published. It threatens their livelihood.Finally, I get to talk about biogerontology, diet and health in one post. For once, I may be doing justice to the blog name. I will also talk about study ethics, particularly the balance of harms and benefits you may expose participants to. In jargon: clinical equipoise.
First, a summary of the study and its results:
N Engl J Med. 2013 Apr 4;368(14):1279-90. doi: 10.1056/NEJMoa1200303. Epub 2013 Feb 25.
Primary prevention of cardiovascular disease with a Mediterranean diet.
Estruch et al.
BACKGROUND: Observational cohort studies and a secondary prevention trial have shown an inverse association between adherence to the Mediterranean diet and cardiovascular risk. We conducted a randomized trial of this diet pattern for the primary prevention of cardiovascular events.
METHODS: In a multicenter trial in Spain, we randomly assigned participants who were at high cardiovascular risk, but with no cardiovascular disease at enrollment, to one of three diets: a Mediterranean diet supplemented with extra-virgin olive oil [MED/EVOO], a Mediterranean diet [MED/nuts] supplemented with mixed nuts, or a control diet (advice to reduce dietary fat). Participants received quarterly individual and group educational sessions and, depending on group assignment, free provision of extra-virgin olive oil, mixed nuts, or small nonfood gifts. The primary end point was the rate of major cardiovascular events (myocardial infarction, stroke, or death from cardiovascular causes). On the basis of the results of an interim analysis, the trial was stopped after a median follow-up of 4.8 years.
RESULTS: A total of 7447 persons were enrolled (age range, 55 to 80 years); 57% were women. The two Mediterranean-diet groups had good adherence to the intervention, according to self-reported intake and biomarker analyses. A primary end-point event occurred in 288 participants. The multivariable-adjusted hazard ratios were 0.70 (95% confidence interval [CI], 0.54 to 0.92) and 0.72 (95% CI, 0.54 to 0.96) for the group assigned to a Mediterranean diet with extra-virgin olive oil (96 events) and the group assigned to a Mediterranean diet with nuts (83 events), respectively, versus the control group (109 events). No diet-related adverse effects were reported.
CONCLUSIONS: Among persons at high cardiovascular risk, a Mediterranean diet supplemented with extra-virgin olive oil or nuts reduced the incidence of major cardiovascular events. (Funded by the Spanish government's Instituto de Salud Carlos III and others; Controlled-Trials.com number, ISRCTN35739639.).Cliffnotes:
I believe the study to be well-designed - despite some short comings - and I do consider it pretty definitive results in favor of the Mediterranean diet and olive oil in primary prevention. Now it is time to act for governments!
And some more details and cliffnotes:
Estruch et al. 2013, n=7447, parallel group, multi-center, randomized, single-blinded trial, 55-80yo, 57% women, n=288 events, median follow-up 4.8yrs (interquartile range, 2.8 to 5.8), ITT+COX regression against confounding, primary prevention/high baseline risk and well-medicated patients.
even more: http://www.wikijournalclub.org/wiki/PREDIMED#cite_note-editorial-4
Now on to the questions and claims regarding PREDIMED:
I hope the answers are helpful to laypeople and experts alike.
#5: Trial stopped early, flawed ethics? ("a biogerontologist's look at the data")
1: Are there large differences between study groups?
“the protocol for the control group was changed
halfway through the trial. The lower intensity of dietary intervention for the
control group during the first few years might
have caused a bias toward a benefit in the two Mediterranean-diet groups,
since the participants in these two groups received a more intensive
intervention during that time.”
“However, we found no significant interaction
between the period of trial enrollment (before vs. after the protocol change)
and the benefit in the Mediterranean-diet groups.”
Additionally,
there is no evidence from the Food Frequency Questionnaires (FFQs) that diet was altered much at all:
“The interventions were intended to improve the overall dietary pattern, but the major between-group differences involved the supplemental items. Thus, extra-virgin olive oil and nuts were probably responsible for most of the observed benefits of the Mediterranean diets. Differences were also observed for fish and legumes but not for other food groups. The small between-group differences in the diets during the trial are probably due to the facts that for most trial participants the baseline diet was similar to the trial Mediterranean diet and that the control group was given recommendations for a healthy diet”
“The interventions were intended to improve the overall dietary pattern, but the major between-group differences involved the supplemental items. Thus, extra-virgin olive oil and nuts were probably responsible for most of the observed benefits of the Mediterranean diets. Differences were also observed for fish and legumes but not for other food groups. The small between-group differences in the diets during the trial are probably due to the facts that for most trial participants the baseline diet was similar to the trial Mediterranean diet and that the control group was given recommendations for a healthy diet”
In fact
(from the interview with Dr. Salas-Salvadó, ref. 1c):
“…after we increased our efforts to foster the
adherence to the low-fat diet (beginning Oct 2006) the advantage of the
fat-rich Mediterranean diet was higher, with a hazard ratio much more
impressive (0.49, 95%CI: 0.26 to 0.92) than when we did less efforts to promote
the low-fat diet (0.77, 95%CI: 0.59-1.00). [p=0.21]”
From this
we can conclude that healthy eating patterns did not change due to counseling,
and if they changed, the effects were modest and involved “Mediterranean” foods
(legumes, fish, wine, sofrito, etc.)
Neither did
physical activity change, making a
bias towards the MED group highly unlikely, since health behavior is
correlated. To give an example: if people did not eat healthier and get more
active in the MED group, why should they have stopped smoking or become more healthier ovarall?
(Additionally,
smoking can be ruled out as a confounder since the diet worked well in
non-smokers, as per subgroup analysis/Fig. 2)
Nonetheless,
one could speculate, that non-food
indices of health may have improved more in the MED group. I would like to
see an analysis of health behavior – even if it is post-hoc.
2: Losses to follow-up
“we had losses to follow-up, predominantly in
the control group, but the participants who dropped out had a worse
cardiovascular risk profile at baseline than those who remained in the study, suggesting a bias toward a benefit in the
control group.”
Overall
losses were very small at 5-12%.
3: Generalizability, “it only worked in the
unhealthy”-type claims (external validity)
“…the generalizability of our findings is limited because all the study participants lived in a Mediterranean country and were at high cardiovascular risk; whether the results can be generalized to persons at lower risk or to other settings requires further research.”
“…the generalizability of our findings is limited because all the study participants lived in a Mediterranean country and were at high cardiovascular risk; whether the results can be generalized to persons at lower risk or to other settings requires further research.”
The subgroup
analysis indeed suggests little benefit in low risk individuals, but given the
confidence intervals a benefit cannot be excluded. A sub-study found improved
cIMT (=a powerful marker of atherosclerosis) in high risk subjects only. I
believe the totality of evidence still suggests that the diet will have
benefits in the healthy, but they will be smaller (obviously) and take longer to accrue.
Adherence
will be the biggest issue whenever the MED-diet is not the traditional diet.
4: They did not test a low fat diet in the
control group
This makes
it all the more impressive. Other RCTs tested low fat diets and showed them to
be quite useless (e.g. WHI, WHEL studies), but this could have been due to
compliance (WHI). All in all, the evidence does not favor low fat diets, see e.g.
Jakobsen et al. 2009 (PMID: 19211817). Since participants were consuming a
Mediterranean diet and olive oil at baseline, diminishing returns may actually bias the result towards the null.
5: Trial stopped early, flawed ethics?
Stopping
too early may indeed exaggerate the benefits of their intervention (as per ref. 1e).
Not only
that. Although, I
have not evaluated all the data I have a very strong suspicion that stopping
the MED/Nuts group, if not both groups, was a huge mistake. Why? This is not a simple concept. When I was first heard this idea, I couldn't believe that we are getting clinical equipoise that wrong (and in most other studies as well); now I am leaning that way. Mea culpa.
First of all, it can be argued that in such a large study clinical equipoise is never violated until all-cause or cause-specific mortality reaches significance. Don’t tell me it’s power limited. Sure statistical power is an issue, but it is not insurmountable. We could easily use a relaxed standard of p=0.10 or 0.1x combined with a highly significant primary endpoint, or increase power through longer follow-up. The MED/EVOO group (almost?) met this standard. MED/Nuts failed miserably.
First of all, it can be argued that in such a large study clinical equipoise is never violated until all-cause or cause-specific mortality reaches significance. Don’t tell me it’s power limited. Sure statistical power is an issue, but it is not insurmountable. We could easily use a relaxed standard of p=0.10 or 0.1x combined with a highly significant primary endpoint, or increase power through longer follow-up. The MED/EVOO group (almost?) met this standard. MED/Nuts failed miserably.
From the get go, one could argue that a huge long-term
study is most ethical. “One study to rule them all”, if you will.
Just look
at the fish oil/n3 fiasco, as one example. How many studies have there now been, some of them
certainly terminated early due to "ethical" concerns? (JELIS, GISSI?) Some of
them much too short (e.g. OMEGA).
Despite all the hype, meta-analyses failed to confirm benefits in secondary prevention and it is very possible that early studies exaggerated the benefits, possibly due to early termination as one reason, so that millions of dollars were sunk into follow-up research, human capital was wasted and people may have been exposed to side-effects from n3 fatty acids.
Despite all the hype, meta-analyses failed to confirm benefits in secondary prevention and it is very possible that early studies exaggerated the benefits, possibly due to early termination as one reason, so that millions of dollars were sunk into follow-up research, human capital was wasted and people may have been exposed to side-effects from n3 fatty acids.
Is this
ethical and just? Is it an efficient use of resources?
What other
reasons were there to continue the PREDIMED study?
There is serious concern that high polyunsaturated fatty acid (PUFA) intakes could be problematic:
pro-inflammatory, pro-oxidative, carcinogenic or suboptimal for other reasons,
offsetting some of their lipid-lowering benefits.
The cancer argument. Equipoise was never in danger,
because the trial was too short to study tumorigenesis and there remains
genuine uncertainty as to which treatment might affect cancer incidence. As stated, PUFA may increase tumorigenesis.
Although, one could argue that CVD benefits are usually enough to terminate a study.
The biogerontologic argument. We know that aging is the prime
cause of CVD, cancer and other unpleasant diseases. As such it is a much more
important field of study than either disease in isolation (Longevity Dividend, Olshansky et al.)
As it so
happens, several aging theories revolve around fat or rather membrane fatty
acid composition. Participants in this study were on average 67 years old,
meaning the study easily could have been extended to look into effects on
aging. I fear money was as limiting as flawed thinking in this case.
If the
study is considered in this light, the actual balance of harms and benefits
would remain unknown.
6: The control diet was unhealthy
I have seen
this claim repeatedly, but it's wrong.
Just as an
example, Ornish (1d) writes:
“the researchers appear to have done everything they could to bias the outcome in favor of the Mediterranean diet by encouraging the "low-fat" diet to increase consumption of foods that are known to increase the risk of cardiovascular disease, including bread, potatoes, pasta, and rice, and not to limit their intake of sodas (which also increase cardiovascular disease risk). See Table 1.”
“the researchers appear to have done everything they could to bias the outcome in favor of the Mediterranean diet by encouraging the "low-fat" diet to increase consumption of foods that are known to increase the risk of cardiovascular disease, including bread, potatoes, pasta, and rice, and not to limit their intake of sodas (which also increase cardiovascular disease risk). See Table 1.”
The latter
part of his assertion is mostly true and irrelevant. The data clearly shows
very little change in these items, i.e. low adherence to low fat
recommendations. Soda consumption did not differ between groups at all! (table S5)
Additionally, the
baseline diet was quite healthy with high fruit/veg, legume and fish intakes.
From the
supplementary data:
“In the Control group, advice on vegetables,
red meat and processed meats, high-fat
dairy products, and sweets concurred with the recommendations of the Mediterranean
diet, but use of olive oil for cooking and dressing and consumption of nuts,
fatty meats, sausages, and fatty fish were discouraged.”
Concerning
the rest of his assertions, imputing malice on the part of the researchers, I am glad that I am not writing for some "respectable" site like the New York Times. I can
use real language on this blog. Fuck you, Dean Ornish. Stop lying.
7: Total mortality did not fall
Mortality
did not fall significantly. In most
cases this is a power issue. In this study a clear trend was evident for the
MED/EVOO group, but not the MED/nuts group. So is EVOO superior to nuts? Now, this
could easily fuel lots of reasonable speculation, given the data accumulated by
Ramsden et al. showing that polyunsaturates can be problematic.
Unfortunately,
detailed mortality data is confidential, because the authors are working on a
new paper. Personally, I do not consider the secrecy necessary, but we will have to wait.
8: The results are not significant
This is
untrue, plain and simple. The primary endpoint is significant. This is the gold standard in clinical research.
9: The effect of the diet is weak; is efficacy
being over- or underestimated?
Example: “…tiny risk reductions…At
the end of the study, when we see an approximately 0.7% ABSOLUTE risk reduction
in the composite endpoint (look at the raw numbers of events, and don't be
fooled by person-years and relative reductions in risk), it seems that such a
small risk reduction could conceivably be entirely due to the major
co-intervention bias - the counseling.” (NEJM, online comment)
This boils
down to the use of relative vs. absolute rates and how you can cheat with
either. Obviously, you always want to look at both. Those who do not argue in
good faith often use absolute numbers to downplay study results. What do the
numbers say? Let’s take a look:
“About 600,000 people die of heart disease in
the United States every year” (cdc.gov)
Let us
scale this up to 6 Million for the world as a whole. The RR for CVD death is
0.70 in the MED/EVOO group, consistent with the primary endpoint. If we work
with just a conservative 10% reduction in relative risk this could save ~60’000 lives in the USA
and at least ~600’000 worldwide.
There is
also another way to look at it:
“Taking into account the small differences in
the accrual of person-years among the three groups, the respective rates of the
primary end point were 8.1, 8.0, and 11.2 per 1000 person-years” (crude rates)
1000 people
need to be treated for a year to prevent 3 major events. That is a NNT of 333/year or 66 over 5 years.
That is a respectable number given that the Mediterranean diet has no (known)
side-effects and may be cost-neutral (i.e. free) if it replaces the regular
diet.
Here, as a
comparison 5-year NNTs for a few common drugs:
“5-year NNT values previously reported in
primary prevention for the use of statins among hyperlipidemic men (5-year NNT,
40 to 70 [multiply with 5 to get one year NNT]), for antihypertensive therapy (5-year NNT, 80 to
160), or for aspirin (5-year NNT, >300)”
Factors exaggerating the benefits: early termination, differences in
intervention-strength ("co-intervention bias"), morbid patients
F. leading to underestimation of the benefits: drop out distribution, diminishing
returns, well-medicated patients (fewer events than expected), potential weight
gain in the intervention groups
All in all,
these factors do not seem to be unbalanced.
10: The Methods are weak
The opposite is true. The study is free from the usual biases we see in the literature: Size (small study, publication bias; this study is large), funding (the study was independently funded, EVOO/nuts were gifts), drop-out rates (exceptionally low), compliance (confirmed using independent methods), missing data, blinding, randomization (in this study randomization was performed according to CONSORT recommendations) or soft endpoints.
The opposite is true. The study is free from the usual biases we see in the literature: Size (small study, publication bias; this study is large), funding (the study was independently funded, EVOO/nuts were gifts), drop-out rates (exceptionally low), compliance (confirmed using independent methods), missing data, blinding, randomization (in this study randomization was performed according to CONSORT recommendations) or soft endpoints.
11: They did not study the MED diet, did they?
This is
arguable. Although, the dietary pattern in the intervention group clearly
shifted to a MED-style diet, in almost all regards, changes were very modest.
This may
have been a study of just isolated EVOO and nuts! Either way, the data supports
MED-style diets.
12: A particular food group is solely
responsible for the benefits e.g. fish
This is
implausible given the tiny dietary changes and earlier research on the MED
diet. There may be one exception: fatty fish. Omega 3 (n3) fatty acid intakes increased modestly
by 0.1-0.12g in the intervention group, but recent epidemiologic data suggests
this could be sufficient and quite low daily intakes may be beneficial. On the
other hand, the n3-CVD link may not even be causal, as recent evidence from
interventional studies has suggested - a study design more powerful than epidemiology.
Additionally,
the baseline intake was 07-0.9 g/d of n3 (table S7), which is above the
recommended “optimal” intake of 0.2-0.5g based on the epidemiology.
13: Low event rates somehow bias the data? Are
these event rates plausible in general e.g. excessive stroke rates?
“the rate of primary events was surprisingly
low for such a high risk group, and because the study was stopped early,
absurdly for “ethical reasons”, the number of events is quite low.”
Is the
event rate (n=288) surprisingly or just expectedly low? We know from experience
that in such studies rates tend to be lower than anticipated.
If my
calculations are correct the actual event rate was at approx. 4% over 5 years.
Whereas a rate of 11% over 6 years “could be conservatively assumed” beforehand (6) but
the authors made room for even lower rates (study protocol, p. 30).
In any case, I am not sure low
event rates have any implications beyond power? And if it affects the latter
this should not matter since 1. power calculations were adjusted accordingly
and the study is clearly powered to detect a 30% relative risk reduction (read
the protocol!) 2. this would provoke type II errors (false negatives).
I am hard pressed to believe the authors missed any events given their protocol, since they used four ways to ascertain events.
I am hard pressed to believe the authors missed any events given their protocol, since they used four ways to ascertain events.
Event rates
could be artificially low if the data was fudged, however, I have not seen any evidence of this.
Another
criticism I have read was that the stroke
rate may be too high, but this was expected from the beginning.
“However, the incidence of stroke in the
Spanish population of these ages is substantial, and almost approximates the
incidence of CHD” (protocol,
version 1, p. 24)
14: How can both EVOO and nuts provide exactly the
same benefits? This seems fishy, no?
I do not
think they provide the same benefits. If you look closely, the RRs of the two treatments
vary and it is just the composite endpoint that is similar. Nuts seem(!) better for stroke, EVOO for mortality. I have not seen
statistical tests comparing the two arms. Unfortunately, the study may be
underpowered for these tests.
15: Why was there no change in the intake of
meds if the participants got healthier?
I can only
guess that the study was not powered to detect such changes. In addition, the
benefits of the MED diet may be mediated by non-classical risk factors. An
earlier report from the PREDIMED study found that classical risk factors
decreased slightly in the intervention groups and Framingham risk decreased
only by approx. 1.7% (-0.2 to -3.3%) (7), which is not enough to explain the
current results.
16: Is the study double-blind or single-blind?
Normally, quality studies of
pharmaceuticals are double or triple-blind. Patients must be blinded to group
assignment, care providers must not know their patients’ group and those assessing
outcomes must also be blinded. Nutrition studies are different.
In a
nutrition study you could blind the primary care physician, dietitian*,
investigators doing the assessment and the participants*.
Unfortunately,
blinding dietitians* and participants* is almost impossible in a study design
with food provision and dietary counseling.
Investigators
were blinded, however: “All medical
records related to end points were examined by the end-point adjudication
committee, whose members were unaware of the study-group assignments.”
So the
study is best described as single-blind – just like the legendary Lyon Diet
Heart Study and many similar trials.
Additionally
(6): “At baseline, general practitioners
(GPs) were not informed of the allocation of participants.”
Does the
lack of double-blinding have any influence on study results?
Physicians
were not involved in carrying out the study. Lack of blinding would be relevant
only if patients told their doctors the group they were assigned to, and if
this systematically changed their treatment. Many ifs.
Some people
will scream bias, but I just don’t see it. If patients thought they are on the
healthier diet they could slack off ("health halo" effect), but they might also
receive better, more aggressive treatment by their doctors if these believe
them to be on a healthy diet; or less treatment, but this is even more doubtful.
17: Why was the primary endpoint changed
during the study and no one told us?
This may be
a case of bad communication. As far as I can tell, when the authors recalculated their sample size based on new
data, they changed the endpoint to approximate the ALLHAT study, which they had
used for power calculations ("Amendments to the Research Protocol", p. 29ff). All of this is standard procedure and is done
before looking at the results(!), but the authors simply failed to spell it out more clearly in their paper: They should have stated that
“therefore the primary endpoint was modified”.
If we look at the sensitivity
analysis we find something approaching the primary endpoint as defined
initially (= current primary endpoint “including
angina plus revascularization”, n = 306 events included):
MED/EVOO: 0.74 (0.57-0.97) p=0.028
MED/nuts: 0.77 (0.58-1.01), p=0.062
This is broadly consistent with all other data.
18: Does
the effect size decrease over time?
It certainly seems so if you look at the sensitivity analysis.
However, the confidence intervals and small
number of cases (N=89) does not justify strong conclusions. Furthermore, I am
not aware of basic research to support a biologic basis of such a phenomenon.
References
1. Estruch, R., Ros, E., Salas-Salvadó, J., Covas, M., Corella, D., Arós, F., Gómez-Gracia, E., Ruiz-Gutiérrez, V., Fiol, M., Lapetra, J., Lamuela-Raventos, R., Serra-Majem, L., Pintó, X., Basora, J., Muñoz, M., Sorlí, J., Martínez, J., & Martínez-González, M. (2013).
Primary Prevention of Cardiovascular Disease with a Mediterranean Diet New England Journal of Medicine, 368 (14), 1279-1290 DOI: 10.1056/NEJMoa1200303
Primary Prevention of Cardiovascular Disease with a Mediterranean Diet New England Journal of Medicine, 368 (14), 1279-1290 DOI: 10.1056/NEJMoa1200303
1c.
Interview with one of the researchers:
http://www.pronutritionistblog.com/predimed-investigator-jordi-salas-salvado-responds-ornishs-critique-and-more/
http://www.pronutritionistblog.com/predimed-investigator-jordi-salas-salvado-responds-ornishs-critique-and-more/
1d. Does a
Mediterranean Diet Really Beat Low-Fat for Heart Health?
1e.
Appel,
Lawrence J., Van Horn, Linda, . (2013) Did the PREDIMED Trial Test a
Mediterranean Diet?. New England Journal of Medicine 368:14, 1353-1354
2.Reduction
in systemic and VLDL triacylglycerol concentration after a 3-month Mediterranean-style diet in
high-cardiovascular-risk subjects
3. Covas
MI, et al. Effect of a traditional Mediterranean diet on lipoprotein oxidation:
A randomized, controlled trial. European Atherosclerosis Society 76th Congress;
June 11, 2007; Helsinki, Finland. http://www.theheart.org/article/796021.do
4. Diabetes
Care. 2011 Jan;34(1):14-9. doi: 10.2337/dc10-1288. Epub 2010 Oct 7.
Reduction
in the incidence of type 2 diabetes with the Mediterranean diet: results of the
PREDIMED-Reus nutrition intervention randomized trial.
Salas-Salvadó
J, Bulló M, Babio N, Martínez-González MÁ, Ibarrola-Jurado N, Basora J, Estruch
R, Covas MI, Corella D, Arós F, Ruiz-Gutiérrez V, Ros E; PREDIMED Study
Investigators.
5. Atherosclerosis. 2011 Nov;219(1):158-62. doi:
10.1016/j.atherosclerosis.2011.06.050. Epub 2011 Jul 6.
Carotid intima-media
thickness changes with Mediterranean diet: a randomized trial
(PREDIMED-Navarra).
Murie-Fernandez
M, Irimia P, Toledo E, Martínez-Vila E, Buil-Cosiales P, Serrano-Martínez M,
Ruiz-Gutiérrez V, Ros E, Estruch R, Martínez-González MÁ; PREDIMED Investigators.
6. Int J
Epidemiol. 2012 Apr;41(2):377-85. doi: 10.1093/ije/dyq250. Epub 2010 Dec 20.
Cohort
profile: design and methods of the PREDIMED study.
Martínez-González
…PREDIMED Study Investigators.
7. Ann
Intern Med. 2006 Jul 4;145(1):1-11.
Effects of
a Mediterranean-style diet on cardiovascular risk factors: a randomized trial.
Estruch et
al.
Just saw this. WOW, Kismet: great, in-depth analysis!
AntwortenLöschen