Are there ‘major doubts’ about the safety of keto?

Another day, another scary news story… Last Thursday, Business Insider whipped up concern with this catchy headline:

• 3 huge new studies of more than half a million people are casting major doubts on the keto diet

But are there really three huge new studies “casting major doubts” about keto and low carb? Let’s take a look.

Study #1: Prospective cohort study published in Lancet Public Health
Surprise! This is the same poor quality observational study for which Business Insider wrote a similar catchy headline in August. We wrote about the problems with this study last month. As did Nina Teicholz, with her op-ed in The Wall Street Journal. As did Dr. George Ede, in Psychology Today. As did functional medicine practitioner Chris Kresser, at length on his blog. If you prefer live TV, prominent cardiologist debunked this study’s claims as completely false on the BBC News. The common thread is that there is nothing new or newsworthy about this poorly designed observational study.

Study #2: Prospective cohort study, not yet published
The second study Business Insider cites is not really a study yet. Rather, it is a press release by the European Society of Cardiology about an abstract presented at its annual meeting. The lead author-to-be, Professor Maciej Banach, of the Medical University of Lodz, Poland, crunched through some NHANES data, along with another unidentified but much larger pool of outcomes, and found weak associations. Until we can see the complete study, we do not know enough to write a detailed critique. But much like the problems with the first study, weak observational associations are notoriously unreliable. More on that, below.

Study #3: Prospective cohort study published on PLOS Medicine
This third study (entitled Nutritional quality of food as represented by the FSAm-NPS nutrient profiling system underlying the Nutri-Score label and cancer risk in Europe: Results from the EPIC prospective cohort study) identifies a weak association between junk food consumption and cancer incidence. There are three problems with citing this as evidence that casts “major doubts” on ketogenic living. First of all, junk food is full of carbs, and a ketogenic diet is a low-carb real food diet, not a junk food diet. Secondly, the association was surprisingly weak, with a hazard ratio of just 1.07. This means that the study data shows that the highest consumers of junk food are only 7% more likely to get cancer than the lowest consumers. This association would have to be much more robust — a hazard ratio of 2.0 and up — to be taken seriously as stand-alone evidence. Finally, the authors mention the most obvious problem in the abstract: “The main study limitation is that it was based on an observational cohort using self-reported dietary data obtained through a single baseline food frequency questionnaire…” It is impossible to extract meaning from poor quality data.

The problem with prospective cohort studies
All three of the “huge new studies” promised in the catchy headline are prospective cohort studies, which are observational. No experiment of any sort was conducted; rather, the study authors look back at and analyze existing data. If you don’t fully understand the reason why observational studies are not reliable, you are not alone. Here are three really thoughtful longer posts that take the time to lay out, in detail, the problem with relying this type of science — also called nutritional epidemiology — to establish a causal relationship:

• Science, pseudoscience, nutritional epidemiology and meat
  — Gary Taubes (blog)
• Remember when a glass of wine a day was good for you? Here’s why that changed
  — Popular Science
• You can’t trust what you read about nutrition
  — FiveThirtyEight

The first one might make you feel better about eating meat. The second may — or may not — make you feel better about your keto cocktail. Either way, you’ll learn more about confounders and selection bias and the confusing mess that is nutritional epidemiology.

Because of the inaccurate food frequency questionnaires and the weak associations that often have nothing to do with causation, Stanford scientist Dr. John Ioannidis recently suggested that journalists should no longer report on these types of findings. Perhaps we can hope for more restraint and fewer misleading, scary headlines in the future.

Earlier

Nina Teicholz in WSJ: “Carbs, good for you? Fat chance!”

Should journalists avoid reporting on most food studies?

‘What the Health’ review: health claims backed by no solid evidence

Guides

The science of low carb

A low-carb diet for beginners