The World Turned Upside Down: The Second Low-Carbohydrate Revolution (37 page)

The ITT controversy.

Advocates of ITT see its principles
as
established and may
dismiss a common sense approach as naïve. They usually say that
removing
non-compliers introduces bias by destroying the original randomization.
In
this, they are confusing the process of randomization with the criteria
for
inclusion. If you excluded people with diabetes from your study and
only found
out when the study started that one of the subjects actually had
diabetes, like
the tainted juries in courtroom dramas, you would be required to remove
them
anyway. You would not have included them if their diabetes came out in
the
voir
dire
, so you can't
include them now. Of course, if you don't know whether or not subjects
have
complied, you have to include everybody but that is what was always
done
because you have no choice. That is not the issue here. What happens
when you
know who did and who didn't comply?

The problem is not easily resolved;
statistics is not axiomatic:
there is nothing analogous to the zeroth law (the idea of thermal
equilibrium
on which thermodynamics rests).  All statistics rests on
interpretation
and intuition. If this is not appreciated, if you do not go back to
consideration of exactly what the question is that you are asking, it
is easy
to develop a dogmatic approach and insist on a particular statistic
because it
has become standard.

As
I mentioned above, I had a good deal of trouble getting my original
paper
published and one anonymous reviewer said that "the arguments presented
by
the author may have applied, maybe, ten or fifteen years ago." This
criticism reminded me by Molieère's "Doctor in Spite of Himself:"

Sganarelle is disguised as a
doctor and spouts
medical double-talk with phony Latin, Greek and Hebrew to impress the
client,
Geronte, who is pretty dumb and mostly falls for it but:

Geronte
:
...there is only one thing that bothers me: the location of the liver
and the
heart. It seemed to me that you had them in the wrong place: the heart
is on
the left side but the liver is on the right side.

Sgnarelle
:
Yes. That used to be true but we have changed all that and medicine
uses an
entirely new approach.

Geronte
:
I didn't know that and I beg your pardon for my ignorance.

In the end, it is reasonable that
scientific knowledge be based
on real observations. This has never before been thought to include
data that
was not actually in the experiment. I doubt that
nous avons
changé tout cela
.

Foster, et al: Outcomes
after 2 Years

Described
in
Chapter
4
as the "shot heard 'round the world," Gary
Foster's
2002 study of low-carbohydrate diets found that, after one year, the
diet was
substantially better than a low-fat diet for markers of cardiovascular
disease.
The study, however, emphasized that, on weight loss, the
low-carbohydrate diet
was better at six months but that the diets were "the same at one
year." This,
of course, was an effect, of dwindling adherence in both the
intervention and
control groups although, in fact,
Figure
4-3
showed that
the lipid markers were noticeably better on the low-carb arm even at 1
year.

A follow-up to the landmark one-year
study, "Weight and
Metabolic Outcomes After 2 Years on a Low-Carbohydrate Versus Low-Fat
Diet,"
published in 2010, had a surprisingly limited impact. What was wrong?
The first
paper was revolutionary and, in the current one, the authors explicitly
addressed
the need for including a "comprehensive lifestyle modification program"
since
the original was criticized for simply giving the people on the Atkins
diet a
copy of the popular book. So what could be wrong? The conclusion was
the same.
The low-carbohydrate arm had better outcomes for most CVD risk factors
although, again, there was no difference in weight loss after 2 years.
As
stated in the conclusion "neither dietary fat nor carbohydrate intake
influenced weight loss when combined with a comprehensive lifestyle
intervention." This is, after all, still the party line and should have
been
well-received by establishment nutrition. That should have been a big
win for
the only-calories-count crowd. Why was there so little impact? In fact,
the
paper is still rarely cited.

Figure 23-2
Cover
page of Foster,
et al,
reference
[77]
.

It is likely that the
zeitgeist
is different from what it was 8 years
ago. Strict
scientific standards have suffered tremendous blows. The willingness to
disregard the results of big expensive experiments and to ignore common
sense
are now much more widespread. Everybody knows that in a big comparison
trial,
the low-carbohydrate diet will win in some way and that it is likely
that the
authors will try to put a negative spin on things. As Elizabeth Nabel
put it
after the embarrassing
WHI
Failure
: "nothing's
changed." Low-fat is still the name of the game and the phrase is
common in the
USDA guidelines and hence in the school lunch program of the Healthy
Hunger-Free Kids Act
[108]
endorsed by Michelle Obama.
Press releases on such government programs quote a progression of
"suits" with
maudlin expressions of optimism about how well what I call "Fruits 'N
Vegetables are doing in improving obesity. There are numerous
disclaimers about
good fats, bad fats so that you can't really hold them to anything.
However,
since Foster's first study, numerous low-carb trials have showed it to
be
superior to any competition. Foster's study was, thus, not the
challenge that
the original was. But, in fact, it should have been given some
attention, at
least because it was a follow-up to the landmark first paper. In my
opinion, it
should have gotten more attention because it reached some kind of new
low in
misleading statements. It was the conclusion that "neither dietary fat
nor
carbohydrate intake influenced weight loss."

I admit that I had not read Foster's
paper very carefully
before making the pronouncement that it was not very good. I was
upbraided by a
student for such a rush to judgment. I explained that it is what I do
for a
living. I explained that I usually don't have to spend a lot of time on
a paper
to see the general drift – I could easily see a couple of errors in
methodology, which I describe below but I was probably not totally
convincing.
So I read the paper, which is quite a bit longer than usual. The main
thing
that I was looking for was information on the nutrients that were
actually
consumed since it was their lack of effect that was the main point of
the
paper. The problem is that people feel that they can call anything that
they
want a low-carbohydrate diet and, of course, people really believe that
"assigned to a low-carbohydrate diet" is the same thing as consuming a
low-carbohydrate diet.

"Any reasonably
intelligent high school
student."

In a diet experiment, the food
consumed should be right up
front but I couldn't find it at all. Foster's paper is quite long with
a
tedious Appendix on the lifestyle intervention but I read the whole
thing
carefully. I really did. The data weren't there. I was going to write
to the
authors when I found out – I think through somebody's blog – that this
paper
had been covered in a story in the Los Angeles Times. As reported by
Bob
Kaplan:

Of the 307 participants
enrolled in the study, not
one had their food intake recorded or analyzed by investigators. The
authors
did not monitor, chronicle or report any of the subjects' diets. No
meals were
administered by the authors; no meals were eaten in front of
investigators.
There were no self-reports, no questionnaires.

The lead authors, Gary Foster
and James Hill,
explained in separate e-mails that self-reported data are unreliable
and
therefore they didn't collect or analyze any."

I confess to feeling a bit shocked. I
don't like getting
scientific information from the LA Times. How can you say "neither
dietary fat
nor carbohydrate intake influenced weight loss" if you haven't measured
fat or
carbohydrate? If you think that self-reported data is not good, then
you can't
make judgments about what was consumed. And, in fact, the whole
nutrition field
runs on self-reported data. Is all that stuff from the Harvard School
of Public
Health, all those epidemiology studies that rely on food records, to be
chucked
out? (Personally, I would say yes, but not for that reason).

What would have happened if the
authors had actually
measured the relevant data, if they asked what people eat, as Kaplan
put it
"the single most important questio ...that any reasonably intelligent
high school
student would ask?" It's not just bad experimental design. It is a
question of
what is on their mind. Do they not know that it is totally
inappropriate to say
that fat or carbohydrate are not important if they haven't measured it?
They
may be so biased that they don't see what is going on. As I mentioned
in
Chapter 4
,
in the first paper, Foster said in
public that he set out to trash the Atkins diet. Not a good way to do
science.
You are supposed to try to trash your own theory and show that it
survives.

Back to
Intention-to-treat.

Foster's paper is some kind of low
point in bias. It also
shows how misleading ITT can be. Chastised for jumping to conclusions,
I
re-read the paper carefully and the Figures in the paper caught my eye.
The
title of
Figure 2
:
"Predicted absolute mean change
in body weight ...." Predicted? That sounds strange. What about the
data? The
figure shows no difference in change in body weight between groups. It
happens.
Usually, the low-carb diet does quite a bit better but no guarantee of
that.
Figure 3 from the paper (my
Figure
23-3
), however,
indicates changes in triglycerides for the 3, 6, 12 and 24 month time
periods.

Re
duction in
triglycerides is virtually the hallmark of low-carbohydrate diets.
Almost
everybody on such a diet lowers their triglycerides (mine fell to half
of the
original value). In
Figure 23-3
,
the difference
between the two diets (shown by the double-headed arrow) was quite
large at 3
or 6 months, the usual result. However, as the experiment continued,
after 12 months,
triglyceride values got closer and actually came together after 24
months.

Figure
23-3
. Figure 3 from Foster, et al.
[77]
The double-headed arrow represents the difference between the change in
triglycerides on the two indicated arms of the study. The distance
shown by the
arrow is plotted on the Y axis of
Figure 23-4
.

This seemed strange so I realized I
had to find out where
"predicted" came from and that meant reading the Methods section and
particularly the Statistical Analysis on how the data had been handled.
In
general, you usually only read the Methods section in detail if you
think that
there was a problem, or if a new method has been introduced, or if you
want to
repeat the authors' experiment yourself. Large studies like this
usually have a
statistician and they use standard methods whose details may or may not
be
understood by a non-statistician (that's me) and frequently not by most
of the
authors. They are usually accepted at face value and it is assumed that
authors
have adequately explained to the statistician what the question is that
they
want to address. As I kept ploughing through the statistical section, I
found
it increasingly tedious and difficult to read until I hit this passage
(I've
highlighted
the key words):