Arithmetic and Butter

Last Tuesday I gave a talk called “Arithmetic and Butter” at the Quantified Self meeting in Mountain View. I had about 10 slides but this one mattered most:

It shows how fast I did simple arithmetic problems (e.g., 2*0, 9-6, 7*9) before and after I started eating 1/2 stick (60 g) of butter every day. The x axis covers about a year. The butter produced a long-lasting improvement of about 30 msec.

I think the hill shape of the butter function is due to running out of omega-3 in Beijing — my several-months-old flaxseed oil had gone bad, even though it had been frozen. When I returned to Berkeley and got fresh flaxseed oil, my scores improved.

This isn’t animal fat versus no animal fat. Before I was eating lots of butter, I was eating lots of pork fat. It’s one type of animal fat versus another type. Nor is it another example of modern processing = unhealthy. Compared to pork fat, butter is recent.

Most scientists think philosophy of science is irrelevant. Yet this line of research (measuring my arithmetic speed day after day, in hopes of accidental discovery) derived from a philosophy of science, which has two parts. First, scientific progress has a power-law distribution. Each time we collect data, we sample from a power-law-like distribution. Almost all samples produce tiny progress; a very tiny fraction produce great progress. Each time you collect data, in other words, it’s like buying a lottery ticket. I realized that a short easy brain-function test allowed me to buy a large number of lottery tickets at low cost. Second, we underestimate the likelihood of extreme events. Nassim Taleb has argued this about the likelihood of extreme negativeevents (which presumably have a power-law distribution); I’m assuming the same thing about extreme positive events (with a power-law distribution). We undervalue these lottery tickets, in other words. Perhaps all scientists hope for accidental discoveries. I seem to be the first to use a research strategy that relies on accidental discoveries.

In the graph, note that one point (actually, two) is down at 560 msec. This suggests there’s room for improvement.

During the question period, a cardiologist in the audience said something about me killing myself — butter is unhealthy. The usual view.

I said I thought the evidence for the usual view was weak. He said, “The Framingham studies.” That was epidemiology, I said. It is notoriously hard to understand. My data was from something like an experiment. Much easier to understand. (And the Framingham study is a terrible example of the supposed evidence. To quote from it: “In the period between the taking of the diet interviews and the end of the 16-year follow-up, 47 cases of de novo CHD developed in the Diet Study group. The means for all the diet variables measured were practically the same for these cases as for the original cohort at risk.”) He replied that the reduction in heart disease in recent years was more support for the usual view. I said the recent decline in heart disease could have many explanations other than a reduction in animal fat intake. Many things have changed over the last 20 years.

There is epidemiological evidence that saturated fat is bad, yes, but it is not the Framingham study nor the recent decline in heart disease. And it really is difficult to interpret. The butter-is-bad interpretation could easily be wrong. The obvious problem is that, after people are told butter is bad, people who try hard to be healthy avoid butter. And they do a lot of other things, too, to be healthy. So butter consumption ends up confounded with a dozen other variables believed to affect your health. When I was growing up, my parents avoided butter because margarine was much cheaper. So butter consumption is confounded with income, another problem.

My tiny experiment, whatever its problems, was much easier to interpret.

I agree with Matt. Just eyeballing the data it looks to me like there are two processes. 1) Steady improvement (which is pretty neat by its self) 2) After you started eating butter you got much faster, but that improvement diminished with time. This fits a placebo hypothesis: you were affected by the thought of eating the butter, but as time progressed that ‘boost’ went away.
This doesn’t mean that eating butter didn’t improve your time (even if just temporarily) but it’s hard to pin down a mechanism.

What type of significance testing has been/could be done on these data? As a first pass (caveat: I’m no statistician), I digitized the points from the figure and did some model testing. No model more complicated than a straight linear decrease (i.e. 2 free parameters) was supported by these data. The later points are definitely, on average, lower than the first – but this can easily be attributed to steady improvement.
In other words, it’s clear that the response times improved over a year, but it is unclear that butter (or anything else beyond random scatter) had an effect.
With respect, am I missing anything here? It seems that, given the apparent noise in the data, one would need much more extensive sampling on many more subjects – i.e., not a lone pioneer, but a large-scale clinical trial.
As much as I would like to believe that eating butter would make me smarter, in the absence of more evidence, this type of conclusion is not science but (at best) pareidolia.

Hi. Your conclusions are wrong. Here are some reasons why…
1. There is (plausibly) some kind of expectation effect here.
2. You did not control for a practice effect, as far as I can tell. You did math, and you did math over many trials, then you started eating butter. Then you did the math faster. That can be construed as a practice effect. I understand your green lines to be mean performance times (please correct me if I’m wrong), but those do not necessarily denote practice nor do they exclude the effect of practice.
3. You did not control for how much you slept, what else you ate and in what quantities, who you met, what you wore, the temperature of the room, the time of day, or other variables so far as I can tell from your data (please correct me if I’m wrong)… all these things can account for performance variations.
4. You are an interested, biased party. Your results cannot be trusted (whether conscious or not, you want to show a difference; thus, to see whether there is an actual difference, you would have to do the experiment on other people, then extrapolate results using, probably, an F-test (ANOVA) through a 2×2 or 2×3 design, most likely with a wait-list control). Your interpretation is thus not as easy as you say, because there are so, so many confounding variables here.
5. If you want to show two things to be different via several trials, you need to show that the difference is significant. Yes, the green lines appear to be different, but they are means (averages, from what I gather — you have no figure legend for your graph), and different means do not actually mean the data are actually different.
6. What are we talking about here, exactly? Saturated fat? Dairy? Because I don’t think you controlled for either in your diet during the experiment. Dairy and saturated fat, for instance, come from many different sources, not just butter.
I don’t know you, but I’d like you to know that you seem like a nice fellow who does interesting things. I’d also like you to know that this is not science, nor is it scientific. I don’t mean that in a mean way, I’m just stating fact. It certainly sounds science-y, but it’s not science… it’s really just a guy talking about an illusory correlation he’s made about butter and math problems. (Also: did you do the same math problems over and over? Because that would definitely lead to a practice effect.)

Gregor Mendel did OK. From Wikipedia: “Mendel’s experimental results have later been the object of considerable dispute.[11] Fisher analyzed the results of the F2 (second filial) ratio and found them to be implausibly close to the exact ratio of 3 to 1.[13] Only a few would accuse Mendel of scientific malpractice or call it a scientific fraud—reproduction of his experiments has demonstrated the validity of his hypothesis—however, the results have continued to be a mystery for many, though it is often cited as an example of confirmation bias.”

I don’t think that the link between speed and butter is at all implausible.

In the book “The Mood Cure”, Julia Ross argues that saturated fats, especially those found in butter, are good for the production of neurotransmitters and hence mood.

Also, in the book “Good Calories, Bad Calories” by Gary Taubes, he systematically debunks the supposed link between saturated fat and triglyceride levels (and hence the link with cardiovascular disease, diabetes, and other symptoms related to the Metabolic Syndrome.)

Ideally, you wouldn’t know whether you are eating butter or not. Maybe have someone randomly buy either margarine or butter, cut it up into small cubes and wrap it in saran wrap so you won’t know the difference when you use it. Just an idea.

The changes in your cholesterol levels will be a good barometer to how your body responds and see if the experiment needs to stop for health reasons.

Partners

Sponsors

Friends of QS

We recently started a program to invite QS Toolmakers to contribute directly to funding our events. We call this program Friends of QS. If you would like to participate we invite you email us to learn more.