Understanding PREDIMED and the Mediterranean diet: heart-healthy or hype?

I am going to discuss the failings of the PREDIMED study, real and imagined. I will do this in a Question & Answer style. Several questions are inspired by theheart.org and NEJM comments and even statements by the low fat "gurus", Essselstyn, Ornish, McDougall or Pritikin. It is no surprise they are up in arms after the study has been published. It threatens their livelihood.

Finally, I get to talk about biogerontology, diet and health in one post. For once, I may be doing justice to the blog name. I will also talk about study ethics, particularly the balance of harms and benefits you may expose participants to. In jargon: clinical equipoise.

First, a summary of the study and its results:

N Engl J Med. 2013 Apr 4;368(14):1279-90. doi: 10.1056/NEJMoa1200303. Epub 2013 Feb 25.
Primary prevention of cardiovascular disease with a Mediterranean diet.
Estruch et al.

BACKGROUND: Observational cohort studies and a secondary prevention trial have shown an inverse association between adherence to the Mediterranean diet and cardiovascular risk. We conducted a randomized trial of this diet pattern for the primary prevention of cardiovascular events.

METHODS: In a multicenter trial in Spain, we randomly assigned participants who were at high cardiovascular risk, but with no cardiovascular disease at enrollment, to one of three diets: a Mediterranean diet supplemented with extra-virgin olive oil [MED/EVOO], a Mediterranean diet [MED/nuts] supplemented with mixed nuts, or a control diet (advice to reduce dietary fat). Participants received quarterly individual and group educational sessions and, depending on group assignment, free provision of extra-virgin olive oil, mixed nuts, or small nonfood gifts. The primary end point was the rate of major cardiovascular events (myocardial infarction, stroke, or death from cardiovascular causes). On the basis of the results of an interim analysis, the trial was stopped after a median follow-up of 4.8 years.

RESULTS: A total of 7447 persons were enrolled (age range, 55 to 80 years); 57% were women. The two Mediterranean-diet groups had good adherence to the intervention, according to self-reported intake and biomarker analyses. A primary end-point event occurred in 288 participants. The multivariable-adjusted hazard ratios were 0.70 (95% confidence interval [CI], 0.54 to 0.92) and 0.72 (95% CI, 0.54 to 0.96) for the group assigned to a Mediterranean diet with extra-virgin olive oil (96 events) and the group assigned to a Mediterranean diet with nuts (83 events), respectively, versus the control group (109 events). No diet-related adverse effects were reported.

CONCLUSIONS: Among persons at high cardiovascular risk, a Mediterranean diet supplemented with extra-virgin olive oil or nuts reduced the incidence of major cardiovascular events. (Funded by the Spanish government's Instituto de Salud Carlos III and others; Controlled-Trials.com number, ISRCTN35739639.).

Cliffnotes:

I believe the study to be well-designed - despite some short comings - and I do consider it pretty definitive results in favor of the Mediterranean diet and olive oil in primary prevention. Now it is time to act for governments!

And some more details and cliffnotes:

Estruch et al. 2013, n=7447, parallel group, multi-center, randomized, single-blinded trial, 55-80yo, 57% women, n=288 events, median follow-up 4.8yrs (interquartile range, 2.8 to 5.8), ITT+COX regression against confounding, primary prevention/high baseline risk and well-medicated patients.

even more: http://www.wikijournalclub.org/wiki/PREDIMED#cite_note-editorial-4

Now on to the questions and claims regarding PREDIMED:

I hope the answers are helpful to laypeople and experts alike.

#1: Are there large differences between study groups?

#2: Losses to follow-up

#3: Generalizability, “it only worked in the unhealthy”-type claims (external validity)

#4: They did not test a low fat diet in the control group

#5: Trial stopped early, flawed ethics? ("a biogerontologist's look at the data")

#6: The control diet was unhealthy

#7: Total mortality did not fall

#8: The results are not significant

#9: The effect of the diet is weak; is efficacy being over- or underestimated?

#10: The employed methods are weak
 

#11: They did not study the MED diet, did they?

#12: A particular food group is solely responsible for the benefits e.g. fish

#13: Do low event rates somehow bias the data? Are these event rates plausible in general e.g. excessive stroke rates?

#14: How can both EVOO and nuts provide exactly the same benefits? This seems fishy, no?

#15: Why was there no change in the intake of meds if the participants got healthier?

#16: Is the study double-blind or single-blind?

#17: Why was the primary endpoint changed during the study and no one told us?

#18: Does the effect size decrease over time?

1: Are there large differences between study groups?

“the protocol for the control group was changed halfway through the trial. The lower intensity of dietary intervention for the control group during the first few years might have caused a bias toward a benefit in the two Mediterranean-diet groups, since the participants in these two groups received a more intensive intervention during that time.”

“However, we found no significant interaction between the period of trial enrollment (before vs. after the protocol change) and the benefit in the Mediterranean-diet groups.”

Additionally, there is no evidence from the Food Frequency Questionnaires (FFQs) that diet was altered much at all:
“The interventions were intended to improve the overall dietary pattern, but the major between-group differences involved the supplemental items. Thus, extra-virgin olive oil and nuts were probably responsible for most of the observed benefits of the Mediterranean diets. Differences were also observed for fish and legumes but not for other food groups. The small between-group differences in the diets during the trial are probably due to the facts that for most trial participants the baseline diet was similar to the trial Mediterranean diet and that the control group was given recommendations for a healthy diet”

In fact (from the interview with Dr. Salas-Salvadó, ref. 1c):

“…after we increased our efforts to foster the adherence to the low-fat diet (beginning Oct 2006) the advantage of the fat-rich Mediterranean diet was higher, with a hazard ratio much more impressive (0.49, 95%CI: 0.26 to 0.92) than when we did less efforts to promote the low-fat diet (0.77, 95%CI: 0.59-1.00). [p=0.21]”

From this we can conclude that healthy eating patterns did not change due to counseling, and if they changed, the effects were modest and involved “Mediterranean” foods (legumes, fish, wine, sofrito, etc.)

Neither did physical activity change, making a bias towards the MED group highly unlikely, since health behavior is correlated. To give an example: if people did not eat healthier and get more active in the MED group, why should they have stopped smoking or become more healthier ovarall?

(Additionally, smoking can be ruled out as a confounder since the diet worked well in non-smokers, as per subgroup analysis/Fig. 2)

Nonetheless, one could speculate, that non-food indices of health may have improved more in the MED group. I would like to see an analysis of health behavior – even if it is post-hoc.

2: Losses to follow-up

“we had losses to follow-up, predominantly in the control group, but the participants who dropped out had a worse cardiovascular risk profile at baseline than those who remained in the study, suggesting a bias toward a benefit in the control group.”

Overall losses were very small at 5-12%.

3: Generalizability, “it only worked in the unhealthy”-type claims (external validity)

“…the generalizability of our findings is limited because all the study participants lived in a Mediterranean country and were at high cardiovascular risk; whether the results can be generalized to persons at lower risk or to other settings requires further research.”

The subgroup analysis indeed suggests little benefit in low risk individuals, but given the confidence intervals a benefit cannot be excluded. A sub-study found improved cIMT (=a powerful marker of atherosclerosis) in high risk subjects only. I believe the totality of evidence still suggests that the diet will have benefits in the healthy, but they will be smaller (obviously) and take longer to accrue.

Adherence will be the biggest issue whenever the MED-diet is not the traditional diet.

4: They did not test a low fat diet in the control group

This makes it all the more impressive. Other RCTs tested low fat diets and showed them to be quite useless (e.g. WHI, WHEL studies), but this could have been due to compliance (WHI). All in all, the evidence does not favor low fat diets, see e.g. Jakobsen et al. 2009 (PMID: 19211817). Since participants were consuming a Mediterranean diet and olive oil at baseline, diminishing returns may actually bias the result towards the null.

5: Trial stopped early, flawed ethics?

Stopping too early may indeed exaggerate the benefits of their intervention (as per ref. 1e).

Not only that. Although, I have not evaluated all the data I have a very strong suspicion that stopping the MED/Nuts group, if not both groups, was a huge mistake. Why? This is not a simple concept. When I was first heard this idea, I couldn't believe that we are getting clinical equipoise that wrong (and in most other studies as well); now I am leaning that way. Mea culpa.

First of all, it can be argued that in such a large study clinical equipoise is never violated until all-cause or cause-specific mortality reaches significance. Don’t tell me it’s power limited. Sure statistical power is an issue, but it is not insurmountable. We could easily use a relaxed standard of p=0.10 or 0.1x combined with a highly significant primary endpoint, or increase power through longer follow-up. The MED/EVOO group (almost?) met this standard. MED/Nuts failed miserably.

From the get go, one could argue that a huge long-term study is most ethical. “One study to rule them all”, if you will. 

Just look at the fish oil/n3 fiasco, as one example. How many studies have there now been, some of them certainly terminated early due to "ethical" concerns? (JELIS, GISSI?) Some of them much too short (e.g. OMEGA).
Despite all the hype, meta-analyses failed to confirm benefits in secondary prevention and it is very possible that early studies exaggerated the benefits, possibly due to early termination as one reason, so that millions of dollars were sunk into follow-up research, human capital was wasted and people may have been exposed to side-effects from n3 fatty acids.

Is this ethical and just? Is it an efficient use of resources?

What other reasons were there to continue the PREDIMED study?

There is serious concern that high polyunsaturated fatty acid (PUFA) intakes could be problematic: pro-inflammatory, pro-oxidative, carcinogenic or suboptimal for other reasons, offsetting some of their lipid-lowering benefits.
PREDIMED could have been turned into a test of the “PUFA has also harmful effects”-hypothesis, once a tentative difference in mortality between the two intervention arms emerged. It wasn’t. Therefore, one can argue that equipoise was maintained all the way through, because the benefits of nuts are doubtful, and the trial (or at least the MED/nuts arm) was terminated for no good reason.

The cancer argument. Equipoise was never in danger, because the trial was too short to study tumorigenesis and there remains genuine uncertainty as to which treatment might affect cancer incidence. As stated, PUFA may increase tumorigenesis. Although, one could argue that CVD benefits are usually enough to terminate a study.

The biogerontologic argument. We know that aging is the prime cause of CVD, cancer and other unpleasant diseases. As such it is a much more important field of study than either disease in isolation (Longevity Dividend, Olshansky et al.)

As it so happens, several aging theories revolve around fat or rather membrane fatty acid composition. Participants in this study were on average 67 years old, meaning the study easily could have been extended to look into effects on aging. I fear money was as limiting as flawed thinking in this case.

If the study is considered in this light, the actual balance of harms and benefits would remain unknown.

6: The control diet was unhealthy

I have seen this claim repeatedly, but it's wrong.

Just as an example, Ornish (1d) writes:
“the researchers appear to have done everything they could to bias the outcome in favor of the Mediterranean diet by encouraging the "low-fat" diet to increase consumption of foods that are known to increase the risk of cardiovascular disease, including bread, potatoes, pasta, and rice, and not to limit their intake of sodas (which also increase cardiovascular disease risk). See Table 1.”

The latter part of his assertion is mostly true and irrelevant. The data clearly shows very little change in these items, i.e. low adherence to low fat recommendations. Soda consumption did not differ between groups at all! (table S5)

Additionally, the baseline diet was quite healthy with high fruit/veg, legume and fish intakes. 

From the supplementary data:

“In the Control group, advice on vegetables, red meat and processed meats, high-fat

dairy products, and sweets concurred with the recommendations of the Mediterranean diet, but use of olive oil for cooking and dressing and consumption of nuts, fatty meats, sausages, and fatty fish were discouraged.”

Concerning the rest of his assertions, imputing malice on the part of the researchers, I am glad that I am not writing for some "respectable" site like the New York Times. I can use real language on this blog. Fuck you, Dean Ornish. Stop lying.

7: Total mortality did not fall

Mortality did not fall significantly. In most cases this is a power issue. In this study a clear trend was evident for the MED/EVOO group, but not the MED/nuts group. So is EVOO superior to nuts? Now, this could easily fuel lots of reasonable speculation, given the data accumulated by Ramsden et al. showing that polyunsaturates can be problematic.

Unfortunately, detailed mortality data is confidential, because the authors are working on a new paper. Personally, I do not consider the secrecy necessary, but we will have to wait.

8: The results are not significant

This is untrue, plain and simple. The primary endpoint is significant. This is the gold standard in clinical research.

9: The effect of the diet is weak; is efficacy being over- or underestimated?

Example: “…tiny risk reductions…At the end of the study, when we see an approximately 0.7% ABSOLUTE risk reduction in the composite endpoint (look at the raw numbers of events, and don't be fooled by person-years and relative reductions in risk), it seems that such a small risk reduction could conceivably be entirely due to the major co-intervention bias - the counseling.” (NEJM, online comment)

This boils down to the use of relative vs. absolute rates and how you can cheat with either. Obviously, you always want to look at both. Those who do not argue in good faith often use absolute numbers to downplay study results. What do the numbers say? Let’s take a look:

“About 600,000 people die of heart disease in the United States every year” (cdc.gov)

Let us scale this up to 6 Million for the world as a whole. The RR for CVD death is 0.70 in the MED/EVOO group, consistent with the primary endpoint. If we work with just a conservative 10% reduction in relative risk this could save ~60’000 lives in the USA and at least ~600’000 worldwide.

There is also another way to look at it:

“Taking into account the small differences in the accrual of person-years among the three groups, the respective rates of the primary end point were 8.1, 8.0, and 11.2 per 1000 person-years” (crude rates)

1000 people need to be treated for a year to prevent 3 major events. That is a NNT of 333/year or 66 over 5 years. That is a respectable number given that the Mediterranean diet has no (known) side-effects and may be cost-neutral (i.e. free) if it replaces the regular diet.

Here, as a comparison 5-year NNTs for a few common drugs:

“5-year NNT values previously reported in primary prevention for the use of statins among hyperlipidemic men (5-year NNT, 40 to 70 [multiply with 5 to get one year NNT]), for  antihypertensive therapy (5-year NNT, 80 to 160), or for aspirin (5-year NNT, >300)”

http://intl-circoutcomes.ahajournals.org/content/2/6/616.full

Factors exaggerating the benefits: early termination, differences in intervention-strength ("co-intervention bias"), morbid patients

F. leading to underestimation of the benefits: drop out distribution, diminishing returns, well-medicated patients (fewer events than expected), potential weight gain in the intervention groups

All in all, these factors do not seem to be unbalanced.

10: The Methods are weak

The opposite is true. The study is free from the usual biases we see in the literature: Size (small study, publication bias; this study is large), funding (the study was independently funded, EVOO/nuts were gifts), drop-out rates (exceptionally low), compliance (confirmed using independent methods), missing data, blinding, randomization (in this study randomization was performed according to CONSORT recommendations) or soft endpoints.

11: They did not study the MED diet, did they?

This is arguable. Although, the dietary pattern in the intervention group clearly shifted to a MED-style diet, in almost all regards, changes were very modest.

This may have been a study of just isolated EVOO and nuts! Either way, the data supports MED-style diets.

12: A particular food group is solely responsible for the benefits e.g. fish

This is implausible given the tiny dietary changes and earlier research on the MED diet. There may be one exception: fatty fish. Omega 3 (n3) fatty acid intakes increased modestly by 0.1-0.12g in the intervention group, but recent epidemiologic data suggests this could be sufficient and quite low daily intakes may be beneficial. On the other hand, the n3-CVD link may not even be causal, as recent evidence from interventional studies has suggested - a study design more powerful than epidemiology.

Additionally, the baseline intake was 07-0.9 g/d of n3 (table S7), which is above the recommended “optimal” intake of 0.2-0.5g based on the epidemiology.

13: Low event rates somehow bias the data? Are these event rates plausible in general e.g. excessive stroke rates?

“the rate of primary events was surprisingly low for such a high risk group, and because the study was stopped early, absurdly for “ethical reasons”, the number of events is quite low.”

http://scienceblogs.com/denialism/2013/03/01/dont-switch-to-the-mediterranean-diet-just-yet/

Is the event rate (n=288) surprisingly or just expectedly low? We know from experience that in such studies rates tend to be lower than anticipated.

If my calculations are correct the actual event rate was at approx. 4% over 5 years. Whereas a rate of 11% over 6 years “could be conservatively assumed” beforehand (6) but the authors made room for even lower rates (study protocol, p. 30).

In any case, I am not sure low event rates have any implications beyond power? And if it affects the latter this should not matter since 1. power calculations were adjusted accordingly and the study is clearly powered to detect a 30% relative risk reduction (read the protocol!) 2. this would provoke type II errors (false negatives).
I am hard pressed to believe the authors missed any events given their protocol, since they used four ways to ascertain events.

Event rates could be artificially low if the data was fudged, however, I have not seen any evidence of this.

Another criticism I have read was that the stroke rate may be too high, but this was expected from the beginning.

“However, the incidence of stroke in the Spanish population of these ages is substantial, and almost approximates the incidence of CHD” (protocol, version 1, p. 24)

14: How can both EVOO and nuts provide exactly the same benefits? This seems fishy, no?

I do not think they provide the same benefits. If you look closely, the RRs of the two treatments vary and it is just the composite endpoint that is similar. Nuts seem(!) better for stroke, EVOO for mortality. I have not seen statistical tests comparing the two arms. Unfortunately, the study may be underpowered for these tests.

15: Why was there no change in the intake of meds if the participants got healthier?

I can only guess that the study was not powered to detect such changes. In addition, the benefits of the MED diet may be mediated by non-classical risk factors. An earlier report from the PREDIMED study found that classical risk factors decreased slightly in the intervention groups and Framingham risk decreased only by approx. 1.7% (-0.2 to -3.3%) (7), which is not enough to explain the current results.

16: Is the study double-blind or single-blind?

Normally, quality studies of pharmaceuticals are double or triple-blind. Patients must be blinded to group assignment, care providers must not know their patients’ group and those assessing outcomes must also be blinded. Nutrition studies are different.

In a nutrition study you could blind the primary care physician, dietitian*, investigators doing the assessment and the participants*.

Unfortunately, blinding dietitians* and participants* is almost impossible in a study design with food provision and dietary counseling.

Investigators were blinded, however: “All medical records related to end points were examined by the end-point adjudication committee, whose members were unaware of the study-group assignments.”

So the study is best described as single-blind – just like the legendary Lyon Diet Heart Study and many similar trials.

Additionally (6): “At baseline, general practitioners (GPs) were not informed of the allocation of participants.”

Does the lack of double-blinding have any influence on study results?

Physicians were not involved in carrying out the study. Lack of blinding would be relevant only if patients told their doctors the group they were assigned to, and if this systematically changed their treatment. Many ifs.

Some people will scream bias, but I just don’t see it. If patients thought they are on the healthier diet they could slack off ("health halo" effect), but they might also receive better, more aggressive treatment by their doctors if these believe them to be on a healthy diet; or less treatment, but this is even more doubtful.

17: Why was the primary endpoint changed during the study and no one told us?

This may be a case of bad communication. As far as I can tell, when the authors recalculated their sample size based on new data, they changed the endpoint to approximate the ALLHAT study, which they had used for power calculations ("Amendments to the Research Protocol", p. 29ff). All of this is standard procedure and is done before looking at the results(!), but the authors simply failed to spell it out more clearly in their paper: They should have stated that “therefore the primary endpoint was modified”.

If we look at the sensitivity analysis we find something approaching the primary endpoint as defined initially (= current primary endpoint “including angina plus revascularization”, n = 306 events included):

MED/EVOO: 0.74 (0.57-0.97) p=0.028

MED/nuts: 0.77 (0.58-1.01), p=0.062

This is broadly consistent with all other data.

18: Does the effect size decrease over time?

It certainly seems so if you look at the sensitivity analysis.

However, the confidence intervals and small number of cases (N=89) does not justify strong conclusions. Furthermore, I am not aware of basic research to support a biologic basis of such a phenomenon.

References

1. Estruch, R., Ros, E., Salas-Salvadó, J., Covas, M., Corella, D., Arós, F., Gómez-Gracia, E., Ruiz-Gutiérrez, V., Fiol, M., Lapetra, J., Lamuela-Raventos, R., Serra-Majem, L., Pintó, X., Basora, J., Muñoz, M., Sorlí, J., Martínez, J., & Martínez-González, M. (2013).
Primary Prevention of Cardiovascular Disease with a Mediterranean Diet New England Journal of Medicine, 368 (14), 1279-1290 DOI: 10.1056/NEJMoa1200303

1.a. http://www.theheart.org/article/1511175.do

1b. http://www.aerzteblatt.de/nachrichten/53537

1c. Interview with one of the researchers:
http://www.pronutritionistblog.com/predimed-investigator-jordi-salas-salvado-responds-ornishs-critique-and-more/

1d. Does a Mediterranean Diet Really Beat Low-Fat for Heart Health?

http://www.huffingtonpost.com/dr-dean-ornish/mediterranean-diet_b_2755940.html?utm_hp_ref=tw

1e.

Appel, Lawrence J., Van Horn, Linda, . (2013) Did the PREDIMED Trial Test a Mediterranean Diet?. New England Journal of Medicine 368:14, 1353-1354

2.Reduction in systemic and VLDL triacylglycerol concentration after a 3-month  Mediterranean-style diet in high-cardiovascular-risk subjects

3. Covas MI, et al. Effect of a traditional Mediterranean diet on lipoprotein oxidation: A randomized, controlled trial. European Atherosclerosis Society 76th Congress; June 11, 2007; Helsinki, Finland. http://www.theheart.org/article/796021.do

4. Diabetes Care. 2011 Jan;34(1):14-9. doi: 10.2337/dc10-1288. Epub 2010 Oct 7.

Reduction in the incidence of type 2 diabetes with the Mediterranean diet: results of the PREDIMED-Reus nutrition intervention randomized trial.

Salas-Salvadó J, Bulló M, Babio N, Martínez-González MÁ, Ibarrola-Jurado N, Basora J, Estruch R, Covas MI, Corella D, Arós F, Ruiz-Gutiérrez V, Ros E; PREDIMED Study Investigators.

5. Atherosclerosis. 2011 Nov;219(1):158-62. doi: 10.1016/j.atherosclerosis.2011.06.050. Epub 2011 Jul 6.

Carotid intima-media thickness changes with Mediterranean diet: a randomized trial (PREDIMED-Navarra).

Murie-Fernandez M, Irimia P, Toledo E, Martínez-Vila E, Buil-Cosiales P, Serrano-Martínez M, Ruiz-Gutiérrez V, Ros E, Estruch R, Martínez-González MÁ; PREDIMED Investigators.

6. Int J Epidemiol. 2012 Apr;41(2):377-85. doi: 10.1093/ije/dyq250. Epub 2010 Dec 20.

Cohort profile: design and methods of the PREDIMED study.

Martínez-González …PREDIMED Study Investigators.

http://ije.oxfordjournals.org/content/41/2/377.long

7. Ann Intern Med. 2006 Jul 4;145(1):1-11.

Effects of a Mediterranean-style diet on cardiovascular risk factors: a randomized trial.

Estruch et al.