The scoring system isn't biased or subjective, it's just a scoring system for "lower/higher carb levels within a population," which is not something that low carbers are interested in.
There are significant metabolic differences between low carb and lower carb (but still glucose driven) http://ajpendo.physiology.org/cgi/content/short/292/6/E1724

Kate: Your friend gives too much credit to the researchers. Very few people in this study, as far as we know, was following a "low carb diet" as the term is normally understood.
Especially as the study relates to what the participants reported that they ate in 1986.

" and to my eye does not look like there is any correlation between the carb score in each group and smoking habits."
Look harder. The "low carb" men are nearly three times(!) as likely to smoke as their high-carb brethren.

The limitations can be written out in the discussion section all you want, but they don't do anyone any good when the authors make statements like this to the media:
"If people choose to eat less carbs, that means they eat more of something else. That something else is better off being plant-based sources of protein and fat rather than ... animal-based sources," Fung said in an interview.
Fung is being awfully definitive given the limitations that you're so happy he admitted to in the study itself.

Yes, but that's not how the media (or the authors) is spinning it. The study data says that in the general population eating lower carb (not low carb) diet relatively high in meat is associated with some issues.

What's with the hate? Table one is not a "smoking gun". The study sucks in several ways, but it is a typical retrospective, food-frequency, cohort-based study.
Also, I do believe they are looking at deciles of carbohydrate intake in the American population, in order to extrapolate to possible suggestions in the future. I did not see a limitation stated about how "This study does not specifically apply to the fine folks at paleohacks.com"

This is a misguided analysis of the study. Table 1 is not the smoking gun, most nurses in the NHS do not work in hospitals (they're 30-55), the scoring system is not the root of the problem and every scoring system exposes the researchers/designers biases.

Agree fully with your analysis -- and thanks for posting on this (and sharing the data). Just another worthless study from the nutrition dinosaurs at Harvard trying to cover their asses for being wrong for the past 30 years. What assholes!

Cows and their milk do have trans fat, but they are not the only natural source (goats and other gnarly animals have some I believe). The amount is around a gram per serving. However, naturally occurring trans fats are likely not as bad as man-made ones:
http://www.sciencedaily.com/releases/2008/04/080402152140.htm

"How low was low carb" turned out to be an excellent question. The answer - not nearly as low as you might expect:
"The "low carb" dieters range from anywhere between 35 to 65% of their calories coming from carbohydrates, and conversely, between 26-41% of their diet from fat. That is not what I would call a low carb diet. In fact, it's a lowfat or a moderate fat/moderate carb diet," a friend with far more experience than my own elaborated.

I did not mean to imply that the people in the study were farmers. Though I realize, it was easy to interpret my statements that way. I was discussing how a natural high-protein diet would occur. People used to eating that way tend to come from that direction - their mothers, wives, their childhood, etc. It is incredibly difficult to eat a high-protein diet from a vending machine.

The idea that cows have trans fat is new to me. Could somebody confirm this? My knowledge is limited as I am very new to this. However, I have never heard of trans fat being anything other than a laboratory-generated compound. I'd love to hear more.

Um, did you read this? She is implying that it's likely that the low carbers in the study were farmers that ate whole high-quality foods than it was that they were busy health professionals that ate whatever crap was in the cafeteria. Besides that, whole cows have plenty of trans fat :) They are the only natural source.

How low was low carb anyway? I cannot access their data so I don't know. But I can say that when I was younger and eating mostly fast food, my diet probably counted as low carb based on most study metrics.

10
Answers

I wrote about it on my blog. Basically, if you are studying a diet, it helps to actually study people following it instead of retroactively saying "oh it looks like this guy ate less than 200 carbs a day and a bunch of meat, let's put him in the low-carb high-meat group and run some stats." This unwitting person could have been shoveling down slim jims and getting all his carbs from beer and cookies.

OK, now that I have the data, here is the smoking gun:

Do any of these diets look like our diet? Are they even significantly different?

As for Yoak's contention that the low carb people were farmers eating whole animals, remember this is a study on health professionals. How many doctors and nurses are also blue collar farmers? How many of them eat hot dogs in the hospital cafeteria? Apparently most of them, even the extolled veggie low carb group. It says a lot that even the veggie low carbers were eating on average .8 serving of processed meat a day! That's certainly more than I eat!

As for the scoring system, that's the root of the problem and can be found here. Is it a scoring system based on the researcher's bias? I say yes. They invent a pattern they want to be the low carb pattern and try to see how closely people conform to it...whether or not these people were low carbers or not. The score is based on dividing up a population. Sorry, that's not how our diet works- it's not based on lower carb than average, it's a fairly unusual system of eating that deserves to be evaluated on clinical trials of diet participants.

This study is like if you took a group of average college students who have never heard of paleo, made up a list of criteria that was "paleo" like high animal fat content and high vegetable content, divided the group of students into statistical groups based on how "paleo" they were...and then used that data to say paleo is bad.

Agree fully with your analysis -- and thanks for posting on this (and sharing the data). Just another worthless study from the nutrition dinosaurs at Harvard trying to cover their asses for being wrong for the past 30 years. What assholes!

This is a misguided analysis of the study. Table 1 is not the smoking gun, most nurses in the NHS do not work in hospitals (they're 30-55), the scoring system is not the root of the problem and every scoring system exposes the researchers/designers biases.

OK, now there is a link to the main body of the study and I can get a better understanding.

Problem number one, the population was scored using percentage of energy eaten of carbs, protein, and fat. Then the scored people were divided into 10 groups, aka deciles. The problem here is that you could be put in the lowcarb group even if you are eating a ton of carbs, as long as you are eating a larger percentage of calories as fat and protein. This is not really low carb. In fact, the LOWEST decile group of carb consumption was eating a mean intake of 116.7 grams of carb per day. This is not super low carb compared to what paleo typically advocates. In fact, it's probably on the higher side of what most paleos eat. Most people in the low carb group were probably not actually low carb at all, at least not according to Atkins or paleo standards. In fact, some people in the lowest carb decile group may well have been eating way more carbs than some people in the high carb groups! For instance, I could eat a steak and a banana and be in the same carb group quintile as someone who ate 10 steaks and 10 bananas. Or someone who ate 10 steaks and nine bananas would be considered in a lower carb group than me, even though they at 9 times more carb than I did.

The researchers also say they did not consider overall caloric intake levels in any of their multivariate analysies. So they did not control for true lowcarbness or calories in the more robust type of statistical analysies. They do mention they did control for caloric intake in 'secondary analysis' but I couldn't find any further details on that or how or when they did it.

OK, so now on to their stastical results. Much of their results was not statistically significant. COnfidence intervals do not mean much as far a I can tell and are not indicative of statistical significance. What you need to look for is a P value of less than .05 . If it's not smaller than .05, then it is not statistically significant and therefore could be more easily due to chance variation. Much of their correlations do not meet that standards of statistical significance.

Interestingly, in a subgroup of individuals that had blood lipids tested, no correlation was found between total cholesterol, HDL, and LDL. So at least they can't argue that low carb is bad for blood lipids. Also, a statistically significant correlation was found between higher carb (as percentage of calories) intake and triglycerides in the blood. No surprise for us paleo eaters there. Carbs are predictable strong risers of trigs in the blood and this shows even in their weird carb scoring system.

They found no statistical diffs correlated with level of animal fat consumption as percentage of calories.
They did find a statistical correlation between increased vegetable fat consumption and lessening of heart disease. They talk about this in a few places, but then buried further down in discussion, they say "only vegetable protein was associated with a significantly reduced risk [of coronary heart disease] in age-adjusted analyses , and this association became nonsignificant in multivariate analyses. So in other words, when they crunched the numbers to control for confounding factors, the correlation between consumption of vegetable fat as percentage of calories and reduced risk of coronary heart disease disappears. THerefore, they are misleading you when they say they insinuate they found a true correlation. They also never directly compare animal fat low carb groups with veg fat low carb groups, so it's hard to say how they directly compare. However, I think at no point do they give us any real evidence that when proper multivariate analysis is used, there is any statistically significant differences between animal and veg fat intake as percentage of calories as compared with any health issues.

ANyway, I think the main point here, when concerned with our own beliefs about eating healthy and eating lower carb, is that they did not truly group people according to actual carb intake according to amount of carbs actually put into the mouth and eaten, and I think that is the biggest flaw in this study. The second would be insinuating an advantage to plant fat intake even though said statistical significance disappears when more robust statical methods are used.

In fact, I don't think this study tells us much if anything about low carb eating because as far as I can tell, only one of the 10 quintiles was even close to being actually low carb.
-Eva

-The result was only statistically significant for cardiovascular mortality. They fail to mention this in the abstract conclusion.

-They compared highest to lowest deciles, which doesn't tell you much about the sample as a whole.

But mostly:

-A high meat diet, for the average American, is almost surely associated with higher (alcohol, trans fat, etc) than a low meat diet.

-These Harvard School of Public Health dudes always pull this shit, and the conclusion is reported in the media as if it is the law. On the one hand, this dumbs down the diet debate quite a bit, and confuses people when other studies come out. On the other hand, maybe it leads some people to do their own research?

I have not found substantial flaws with the study and I am curious to hear more from those more experienced than I.

Smoking is accounted for in the study - and to my eye does not look like there is any correlation between the carb score in each group and smoking habits. Nor do I expect there to be more or fewer smokers among animal vs. veggie eaters low-carb eaters.

While it is true that the study does not account for meat quality, you would expect it to be higher among those who consume a lot of meat thereby make it an important point in their lives.

"A high meat diet, for the average American, is almost surely associated with higher (alcohol, trans fat, etc) than a low meat diet." - not true. Transfat consumption was higher in the low-carb veggie group. After all, it is difficult to find sources of trans-fat in a cow. Most of them will be coming from processed vegetable foods.

"This unwitting person could have been shoveling down slim jims and getting all his carbs from beer and cookies." - while this may be true in one particular case, it is unlikely to influence a sample as large as this. Low-carb animal meat diets require a lot of cooking from raw ingredients. It is very hard to eat enough in slim jims (not to mention expensive!) and you are far more likely to meet people who live on potato chips than slim jims. A typical unhealthy american does not eat a low-carb diet. Most get 80-90% of their calories from carbohydrates. Low-carb nutirion would naturally occur in farmers, old-fashioned blue color families, those who are simply used to eating a whole cow for dinner. Those who eat slim jims are likely to consume so much of the other kinds of junk, they would not wind up in the low-carb group.

"How low was low carb" turned out to be an excellent question. The answer - not nearly as low as you might expect:
"The "low carb" dieters range from anywhere between 35 to 65% of their calories coming from carbohydrates, and conversely, between 26-41% of their diet from fat. That is not what I would call a low carb diet. In fact, it's a lowfat or a moderate fat/moderate carb diet," a friend with far more experience than my own elaborated.

The idea that cows have trans fat is new to me. Could somebody confirm this? My knowledge is limited as I am very new to this. However, I have never heard of trans fat being anything other than a laboratory-generated compound. I'd love to hear more.

Cows and their milk do have trans fat, but they are not the only natural source (goats and other gnarly animals have some I believe). The amount is around a gram per serving. However, naturally occurring trans fats are likely not as bad as man-made ones:
http://www.sciencedaily.com/releases/2008/04/080402152140.htm

How low was low carb anyway? I cannot access their data so I don't know. But I can say that when I was younger and eating mostly fast food, my diet probably counted as low carb based on most study metrics.

" and to my eye does not look like there is any correlation between the carb score in each group and smoking habits."
Look harder. The "low carb" men are nearly three times(!) as likely to smoke as their high-carb brethren.

I did not mean to imply that the people in the study were farmers. Though I realize, it was easy to interpret my statements that way. I was discussing how a natural high-protein diet would occur. People used to eating that way tend to come from that direction - their mothers, wives, their childhood, etc. It is incredibly difficult to eat a high-protein diet from a vending machine.

Kate: Your friend gives too much credit to the researchers. Very few people in this study, as far as we know, was following a "low carb diet" as the term is normally understood.
Especially as the study relates to what the participants reported that they ate in 1986.

Um, did you read this? She is implying that it's likely that the low carbers in the study were farmers that ate whole high-quality foods than it was that they were busy health professionals that ate whatever crap was in the cafeteria. Besides that, whole cows have plenty of trans fat :) They are the only natural source.

WHen I took statistics at UCLA, they taught us that we were not allowed to count something as 'correlations' unless they were stastically significant. Therefore, in the abstract, we would be required to write, "No correlation was found" for anything that was not statistically significant. To do anything else is to put your own agenda way above truth. Or maybe quality of research has degraded badly in intervening years as I am seeing this kind of thing regularly these days.

Anywho, since I can't access the details of the study setup, it's hard for me to make many useful comments other than to note an apparent bias and agenda to this research. Maybe someone could post the details here? I do think it's an interesting subset that was chosen for this epidemiological study. However, I do have to wonder if this data was not another case of plugging in numbers for hundreds of variables in various ways until you finally find a few that support your theory. As it is, most of the correlations were not significant. One has to wonder how many hundred of numbers were run to get the one for cardio that was statistically significant. How many of the correlations found might have been supportive of contrary hypothesies? We can't know unless the full list of data gathered and used is posted somewhere as well as data on all multivariant analysies done.

I do know that numbers hunting and fishing in epidemiological studies has reached ridiculous levels in recents years. I guess it's a lot easier now that computers can crunch the numbers for you in seconds. Minger's reanalysis of the work in the epic China Study book is rapidly becoming a classic example: http://rawfoodsos.com/ . Given enough numbers, you can usually pick out a few that might give credence to any one idea. THose with an agenda will be happy to only report those few data points. But those seeking the truth will have to look at all the variables in a very open minded way. Did the people doing the study in question look at the data in a fair and openminded way? By their manipulation of the information in abstract, I personally would tend to think not. However, perhaps some of my questions are answered in the main study writeup, if I could access it..

In this study the participant with the lowest carbohydrate intake had a carbohydrate intake (percentage of energy) of 35%. This is nowhere close to the level of carbohydrate restriction in any phase of the Atkins Diet.

The scoring system isn't biased or subjective, it's just a scoring system for "lower/higher carb levels within a population," which is not something that low carbers are interested in.
There are significant metabolic differences between low carb and lower carb (but still glucose driven) http://ajpendo.physiology.org/cgi/content/short/292/6/E1724

Let me just quote from the discussion of the full text to attempt to illustrate what this study looked at.

"In our 2 cohorts of U.S. men and women who were followed for 20 to 26 years, we observed that the overall low-carbohydrate diet score was only weakly associated with all-cause mortality. However, a higher animal low-carbohydrate diet score was associated with higher all-cause and cancer mortality, whereas a higher vegetable low-carbohydrate score was associated with lower mortality, particularly CVD mortality."

"Our study has limitations. The low-carbohydrate diet scores were not designed to mimic any particular versions of low-carbohydrate diets available in the popular literature. Therefore, the risk estimates do not directly translate to the assessment of benefit or risk associated with the popular versions of the diet. In addition, the participants of our cohorts have higher educational status and better availability of health care coverage. Therefore, results may not be directly generalizable to the general population."

Melissa has linked the least important table of the study, download the full text and look at tables 2, 3 & 4 and then draw your own conclusions. http://ifile.it/yqhbzon/289.full.pdf

What's with the hate? Table one is not a "smoking gun". The study sucks in several ways, but it is a typical retrospective, food-frequency, cohort-based study.
Also, I do believe they are looking at deciles of carbohydrate intake in the American population, in order to extrapolate to possible suggestions in the future. I did not see a limitation stated about how "This study does not specifically apply to the fine folks at paleohacks.com"

Yes, but that's not how the media (or the authors) is spinning it. The study data says that in the general population eating lower carb (not low carb) diet relatively high in meat is associated with some issues.

The limitations can be written out in the discussion section all you want, but they don't do anyone any good when the authors make statements like this to the media:
"If people choose to eat less carbs, that means they eat more of something else. That something else is better off being plant-based sources of protein and fat rather than ... animal-based sources," Fung said in an interview.
Fung is being awfully definitive given the limitations that you're so happy he admitted to in the study itself.