How Kevin Hall tried to kill the insulin hypothesis with pure spin

Kevin Hall, the senior NIH researcher recently published a paper in AJCN that has received a lot of media attention. He claims this study refutes the insulin hypothesis so completely that it is now ‘dead.’ That’s interesting, I thought, as I sat down to read the article.
Confirmation bias is a well-known psychological phenomenon whereby facts that agree with your pre-formed opinion are accepted as true and those that are not are ignored. All facts become filtered through this bias to confirm your previously held opinion. It’s also known as a closed mind.
So, let’s take a closer look at this paper and its claims. The paper is titled “Energy expenditure and body composition changes after an isocaloric ketogenic diet in overweight and obese men“. Let me give you some background. The award-winning science journalist Gary Taubes believes that obesity is essentially a disease of too much insulin – hyperinsulinemia. As refined carbohydrates stimulate insulin more than fat or protein, reducing carbs will result in greater fat loss.
Gary Taubes set up the non-profit organization NuSI to raise money to fund research and this paper is the first one published. 17 overweight men were admitted to a metabolic ward where all the food they ate was carefully measured. There was a 4-week run-in phase to establish a baseline where men would eat a high carbohydrate, high-sugar diet and then switch to a carefully designed low carbohydrate, low-sugar diet. Various measurements were then taken, including energy expenditure (EE – how many calories the body is burning), over the next 4 weeks.
Does a ketogenic diet cause fat loss?
Those are all facts, not opinions, derived straight from the study. Isn’t that a good result?
Well, if you’re Kevin Hall, no. You need to find a way to spin this in a negative way. Then you can tell all your friends in the media so you can declare ‘I was right.’ Let’s see how this was done.
When patients embarked on their run-in phase, they were switched to a 2700 calorie/day high-sugar high-carb diet, meant to replicate the Standard American Diet (SAD) that caused the obesity epidemic. Nobody actually believes that this is a healthy diet, and nobody believes it should cause fat loss. But it did. Why?
Anybody who has done research knows why. It’s the effect of going into a study and knowing that people are testing you. It’s a universal effect. That’s the precise reason why we have run-in phases. To establish a baseline.
So, people lost weight on this SAD diet. But instead of using this new baseline, Hall decides that the downward trend is the new baseline. The unspoken premise or assumption is that if these people had taken another 4 weeks of the SAD, they would continue to lose weight at the same rate indefinitely. WHAT? Are you out of your mind? That’s completely illogical.
Let’s take an analogous situation. Suppose we are teaching math. We teach one semester with no tests, no exams, no checking of homework and no projects. Students are just supposed to spend 1 hour in class and 1 hour of homework a day. They all say they do it. Then, unbeknownst to them, we test them on a standardized test. They do really bad and score 65%.
Next semester, they have daily tests, a final exam, and daily checking of homework. They still spend 1 hour in class and 1 hour of homework. Scores should be theoretically unchanged, because they were doing the same amount of work. Of course, in reality this is completely false. Because they know we are regularly checking them, they do a better job. Now they score 80%.
This is the same effect we see when people enter a study. No matter what we are measuring, things improve simply by entering a study. It happens with blood pressure, blood sugars, cholesterol, diets, depression – everything. But the results do not get better indefinitely. It’s a one time benefit.
Students’ scores might improve from 65 to 80 in one semester. This does not mean that another semester of testing will raise their scores to 95. Instead they will likely stay at 80. But this is exactly what Hall does – he assumes that this one-time benefit will persist indefinitely.
By making this assumption that the SAD diet will continue to cause fat loss (which logic tells us is false) you can make a positive result negative. So, yes, KD does cause fat loss, but does not INCREASE fat loss and then you can make this your conclusion. Since most of Hall’s journalist friends never read the paper and only the abstract, it’s easy to convince them.
According to Hall’s assumption, you should therefore simply continue eating the SAD with 25% sugar and expect to lose weight indefinitely. Go ahead. See what happens. I already know. So do you. You’ll get fat, you’ll get type 2 diabetes, and then eventually, I’ll put you on dialysis and chop off your feet when they go gangrenous. But at least Hall can say that he was right.
What about energy expenditure on a ketogenic diet?
Look at how Hall describes the absolutely critical increase in EE. Here’s what he writes:
the KD coincided with increased EEchamber (57 ± 13 kcal/d, P = 0.0004) and SEE (89 ± 14 kcal/d, P < 0.0001) [emphasis mine]
Hall is telling you that this was merely a coincidence that patients are all burning an extra 57 calories per day. WTF??? There is nothing coincidental about it. You switched them to a KD. EE increased. The P value of 0.0004 means that there is a 99.96% chance that this is NOT A COINCIDENCE. Hall knows this as well as I do. This is basic statistics 101. Hall, a mathematician is surely aware of this.
So EE increased and yes, the effect waned over time. What did he expect? That things would continue indefinitely in a straight line? Life doesn’t work that way. Hall had assumed this would happen for fat loss during the SAD, but then correctly points out that EE does not. It doesn’t in either case, dude. Get a clue.
The key to lasting weight loss
The reason why EE is so critically important is that this is the key to lasting weight loss. Hall had just been profiled in the New York Times cover page measuring the EE of Biggest Loser Contestants. The reason they all regained their weight was that their EE slowed to such a degree that caloric reduction could not keep up.
So, an intervention like a ketogenic diet that increases EE is HUGE, GREAT NEWS. Except, of course if you’re Kevin Hall, because it means you were wrong. And you care more about your reputation than people’s health and well-being.
Some people have also noted that this study controlled calories so it negates one of the KD’s biggest advantages, which is that it makes you feel full. Well, sorry, guys, that’s not the question it is designed to answer. Same goes for the fact that there are only 17 people in it. Again, that’s the study design, so it is what it is, and there’s no use complaining about it.
The problem is not the data but the ‘spin’
In the end, the main problem is not the study data. The data are excellent. The problem is the ‘spin’. Here’s the conclusion Hall writes in the abstracts conclusion (which is the single most important few sentences of the paper, the one that everybody reads).
The isocaloric KD
was not accompanied by increased body fat loss but was associated with relatively smallincreases in EEthat were near the limits of detection with the use of state-of-the-art technology.
I’ve highlighted what is fact. I’ve crossed out what is pure spin. Did the KD cause body fat loss? Yes it did. And that’s really, really important. Hall spins it this positive into a negative by moving the goalposts – “Oh but it didn’t do better than before”.
Then he says the increase in EE is “relatively small’. So what? Did it increase or not? In fact, your own study from the Biggest Loser suggests that weight loss will DECREASE EE, so even the stabilization (let alone the increase) of EE is critically important. That’s the gold medal, buddy! You just threw it in the garbage.
Hall then downplays this relationship by calling it an ‘association’. As if the change in EE just happened to occur at the same time as the change in diet. What a load of crap. You changed the diet and measured the change in EE. Nobody doubts that. It’s causation, pure and simple. So why try to spin this as an ‘association’ which is a mere ‘coincidence’? Pure spin.
Hall then further tries to downplay the importance of stable EE by saying it’s ‘near the limits of detection with the use of state-of-the-art technology’. So what? Who cares? Did it stabilize or not? Isn’t that great news? Did you not just show that weight loss efforts fail because of decreased EE?
Unfortunately, spin-doctor Hall is now entering a logic free zone, and many journalists like Julia Belluz and other bloggers are happy to take what is shared at face value. “In this first Biggest Loser study, I demonstrate why stable EE is critical for weight loss. In this second study I’ll show how stable EE is absolutely worthless. Ta Da!”
Hall wants desperately to save his own reputation, even if he has to sacrifice your health to do so. Sad. So sad.
The facts alone, without any spin would be this. A ketogenic diet, independent of calories, causes fat loss and causes an increase (or at least the stabilization) in EE. That’s the facts. And I love it. Because I can use these facts to help heal patients and save lives.
—
Jason Fung
More
A Quick Guide to Ketogenic DietsWhy the First Law of Thermodynamics Is Utterly Irrelevant
How to Fix Your Broken Metabolism by Doing the Exact Opposite
Videos about insulin
- MEMBERS ONLY
- MEMBERS ONLY
More with Dr. Fung
Dr. Fung has his own blog at intensivedietarymanagement.com. He is also active on Twitter.
His book The Obesity Code is available on Amazon.
What is wrong with this picture? I wondered why discussion of this study took so long to have its own article here. Now you say you "had my concerns when Peter Attia left"...
Suggesting he didn't agree with the way the study was being run...
The science is the science. Both sides will try to make it fit their pre-conceived ideas about what the results "should be"... Simply attacking Kevin Hall is bush league and makes the entire NuSi concept a joke.
I surely hope you don't investigate Peter Attia's current dietary habits as found on eatingacademy.com. I'll save you the effort.
"These days I’m eating about as freely as I have in 7 years. I’m still carb-restricted by the standards of most Americans, but nowhere near the ketogenic lines of 2011, 2012, and 2013. I almost always skip breakfast, and lunch is usually a salad (“in a bowl larger than my head,” if possible). Dinner is usually a serving of meat with more salad and veggies. I’m more liberal on fruit and even occasionally rice or potatoes. Also, in moments of weakness I sometimes lean into my kid’s crappy food."
Personally I don't care what the outcome of the study is, as long as the investigators are not biased. I don't suddenly accuse them of bias just because they don't come up with the results that comport with a belief I hold one way or the other.
Kevin Hall, no more biased than Jason Fung.
Future studies will investigate more fully. The science will slowly reveal itself despite the wishes of those on side-A or side-B.
Perhaps someone can post a link to Taubes response to the NuSi study. Hard to imagine he would be attacking the first study released from the organization he co-founded.
I await his response.
The way I see it is pretty simple, as the null hypothesis is this : we shouldn't see any change between a high-carb and a ketogenic diet if they are isocaloric.
Looking at the data, this is not the case. Thus, you have two conclusions possible.
First, you can say that the results are in contradiction with the null hypothesis. Non-coincidental increase of energy expenditure with dietary change is indeed in contradiction with the "a calorie is a calorie" theory.
Or, you can say the results were inconclusive with regards of all the possible biais that could have happened in the study. (That would be my opinion).
The one thing you can't really say is that it proves the null hypothesis. Because the data doesn't point in that direction. So when Jason Fung says Kevin Hall spined the study results in his conclusion, he definitely makes a point.
Let's imagine this study was in the field of physics and it tried to disprove Einstein's theory of general relativity. Unless the data shows with absolute certainty that general relativity is broken, there is no way in hell the researchers could even dream of saying otherwise in their conclusions.
It would be like saying : "Gravitational waves have been detected at LIGO, but their impact was so small it was barely measured by our state-of-the-art interferometer.". In the field of physics, this doesn't happen, as everything is checked a thousand times and the smallest biais or measurement error will get your entire study rejected.
Also, it might be interesting to remember that one of the premises of the ketogenic diet is that there is an adaptation period in which we see a lot of metabolic changes. As far as I'm concerned, the timescale of this study is not long enough by a very long shot to be of interest with regards to ketogenic diets.
What I would like to see is a similar study, with less biases, done on a much longer period - say one year -, and comparing two sets of individuals using different diets. I guess I'm just asking for a randomized, double blind placebo-controlled study.
Actually, NuSI deliberately picked a hostile investigator known to be a CICO zealot (and who has shown some talent for sabotaging LC trials, although perhaps not this one). My understanding is that NuSI did so out of an abundance of concern that any results reported by LCHF-friendly PIs would be disregarded.
re: What is wrong with this picture?
The only thing wrong with the picture is that the actual paper is being held for ransom behind a paywall, so the general public can't read it for themselves. If NuSI doesn't arrange for open access, I'd have to conclude that they seriously bungled that aspect of the initiative.
It's great that credible critics have paid the piper and are sharing their insights, but I'd really like to read it myself.
«…I don't care what the outcome of the study is, as long as the investigators are not biased.»
With a well designed trial, that shouldn't matter, and my understanding is that this over-arching objective was one of the goals.
«Perhaps someone can post a link to Taubes response to the NuSi study.»
There hasn't been one yet, as far as I know, and yes that's a concern, as is the fact that the NuSI web site itself hasn't had much attention for nearly a year.
http://ajcn.nutrition.org.sci-hub.cc/content/early/2016/07/05/ajcn.11...
Hopefully, the follow-on study will be larger, and will have a control group. Take 100 participants, put 50 in the KD arm and 50 in the SAD arm, and follow them for 8 weeks. Everyone gets a DEXA scan at the start. midpoint, and finish, plus the usual tests (blood lipids, insulin, CMP). If Dr. Fung is correct, the weight loss in the SAD group will taper off after a few weeks as the participants adapt, while it will advance greatly in the KD group. 8 weeks of isocaloric diets should allow for a good separation between the KD and SAD groups in terms of fat loss, change in resting energy rate, etc. Ludwig did something similar in his 2012 study, but he did not keep his subjects locked up and closely monitored. In the 2012 Ludwig study, the subjects were followed for a seven-month period, but not locked up in the hospital. This allows some detractors to claim that study participants could cheat on their diets. I just hope the NuSi follow-on study will be designed to address any criticisms, and hopefully give some useful data.
At the end of the day, I think most people just want to know the best diet, be it LCHF, low-fat, Mediterranean, etc. Settling the "a calorie is a calorie" debate will move things in some direction, because there are other things to consider (like, even if KD and SAD/LowFat diets have similar rates of fat loss, are there other benefits?).
Also the guy says that they were losing muscle mass. Is this because they were consuming too much or too little protein? If they did the nitrogen test and it's because they're consuming too much protein it would have nothing to do with breaking down protein from lean tissue i think. If it's from too little protein then all i can say is if you're consuming food like meat from animals or eggs you should automatically be getting enough protein and not have this problem. Most low carbers i know of say moderate to high amounts of protein, but it seems like this study might have given them too low of protein.
So what did this article really consist of... well, other than the blatant appeal to authority over Taubes, and the horrible assumptions of what Hall "implied", the only real point made was that this SINGLE, let me emphasize, ONE STUDY in the face of the entirety of research on the topic, showed that KD resulted in a ~57 kcal increase in TDEE. When it was already established, in the study you brought forth as evidence, virtually no fat loss difference occured. Lmao, 57 calories, so what? Also, how could that not be coincidence? Because an arbitrary equation called p value? That is such a insignificant difference, even if their means of measuring were as accurate as they say, it wouldn't matter whatsoever.
You may be an MD but man, this is just sad. Didn't expect anything more from a Keto zealot.
Nothing.
I'm researching keto for my husband and came across this article.
Because I'm old, I lived the low fat 90s. People dropped weight left and right eating fruit, pretzels, and snackwell cookies. Nobody got diabetes or lost their feet! I'm not saying this is in any way ideal eating, but I keep coming across this assumption in the keto community that it's impossible to lose weight and sustain health on high carb, sub par diets.
This type of medically recommended low fat diet lasted for at least a decade. I remember buying fruit juice and low fat sour cream and thinking these were healthy choices. And most of my peers did the same, we are the same demographic (college educated women) that are enthusiastically choosing paleo low carb diets today. Apparently we should have sustained higher risks weight gain, diabetes, etc but this didn't play out anecdotally and I can't find any data to support what should be a huge spike in poor health in my age/gender group. But it seems education has a protective effect on health in women, and my age group is no different.
Again, low fat high sugar is not ideal, obviously, but given all of the cofounders in dietary science, the effects of eating do not seem to be clear cut, or perhaps they are less dramatic than we would like to think. This study, and the counterpoint here, suggest that to me. A 57 calorie/day (a few almonds) EE is pretty negligible, clearly doesn't sustain. I do agree that attenuation in adherence is a problem for diet studies.
On what planet is increased muscle break down considered a metabolic advantage?
There were 21, not 17 people completing the study.
"A controlled 3-way crossover design involving 21 overweight and obese young adults"
I think "17" refers to the Hall study.
http://ajcn.nutrition.org/content/early/2016/07/05/ajcn.116.133561.ab...
Effects of Dietary Composition on Energy Expenditure During Weight-Loss Maintenance
https://jamanetwork.com/journals/jama/fullarticle/1199154
"REE was 67 kcal/d" versus Hall's averaged 56kcal/d.
Crazy stuff folks.
"wrong according to literally all the current evidence"
Energy Metabolism in the Liver
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4050641/
"Insulin stimulates glycolysis and lipogenesis, but suppresses gluconeogenesis"
Just absolutely crazy non-sense from an illiterate charlatan.
Fantastic. Except in the real world, where people are NOT on calorie-controlled diets, telling them to "eat less" doesn't work because the energy-dense, insulinogenic foods they're eating pack in the calories while turning on their cravings, make it nearly impossible to slowly and effectively decrease their calorie intake.
Keto diets, Intermittent Fasting, etc. are strategies for the REAL WORLD that attempt to fix the UNDERLYING issues that prevent people from being able to "eat less." A less insulinogenic diet is an attempt to decrease energy-dense processed garbage and stabilize appetite in the hopes that it will lead to a decreased calorie intake (as well as improving other health markers, obviously).
Amazing that so few understand this.