The American Academy of Pediatrics (AAP) and the American Heart Association recommend that children drink skim or low-fat milk after age 2. But that may not help them to avoid obesity.

The U.S. Department of Agriculture recommends that everyone older than age 9 consume three cups of dairy, which includes milk, cheese and yogurt, a day. Because whole milk contains a considerable number of calories from fat, however, nutritionists have advised people to pick up skim, 1%-fat or 2%-fat versions instead, with the idea that they provide the same calcium and vitamin D for the bones, but without the weight gain that can increase risk of heart disease and diabetes.

“It is one of the most long-standing and consistent nutritional recommendations in the United States, going back 50 years essentially,” says Dr. David Ludwig, director of the New Balance Foundation Obesity Prevention Center at Boston Children’s Hospital.

But is it based on solid scientific evidence? In an editorial in the journal JAMA Pediatrics, Ludwig and nutrition expert Dr. Walter Willett, chair of the department of nutrition and epidemiology at Harvard School of Public Health, argue that there is actually little data to support the idea that skim and low-fat milk lead to better health outcomes than whole milk. Here are a few of their reasons.

1. Low-fat foods do not lower calorie consumption: Low-fat versions are supposed to reduce the amount of calories that people eat, and in an absolute sense, they do. A cup of low-fat milk contains fewer calories than a cup of whole milk. But Ludwig and Willett note that there isn’t much evidence to support the idea that drinking lower-calorie beverages in general leads to lower-calorie intake. Reduced-fat foods and drinks may not be as filling, so consumers may end up compensating for the lack of calories and eating or drinking more. In a study published in the Archives of Disease in Childhood in March, scientists found that kids who drank lower-fat milks were actually more likely to be overweight later on.

“Our original hypothesis was that children who drank high-fat milk, either whole milk or 2% would be heavier because they were consuming more saturated-fat calories. We were really surprised when we looked at the data and it was very clear that within every ethnicity and every socioeconomic strata, that it was actually the opposite, that children who drank skim milk and 1% were heavier than those who drank 2% and whole,” study author Dr. Mark Daniel DeBoer, an associate professor of pediatric endocrinology at the University of Virginia School of Medicine and the chair-elect for the AAP Committee on Nutrition, told TIME in March.

Ludwig’s other research on food addictions support the theory that increased levels of certain foods with a high-glycemic index, such as bagels, white rice and instant oatmeal, can increase hunger and contribute to weight gain, but drinking whole milk was not associated with packing on the pounds.

2. Low-fat milk increases levels of triglyceride fats: Whole milk’s high saturated-fat content has been linked to higher cholesterol. But low-fat milk may lead drinkers to consume more high-glycemic-index foods, which can increase the level of triglycerides that can amplify the effect of heart-disease risk factors such as high cholesterol and hypertension.

3. The rise of sugar-sweetened flavored milk: Some schools and health experts condone flavored milk as a way to get kids to drink it — and the kids do. But while these versions may have 3 g less saturated fat, they also contain about 13 g more sugar than whole milk per cup.

“We are not arguing milk should be abandoned,” says Ludwig. “For kids who are consuming diets that are high in calories and low in nutrients, milk can provide lots of needed nutrients — like calcium.” What he and Willett are addressing is the assumption, driven by seemingly sound logic, that low-fat versions of milk are healthier than whole milk. “Somehow this low-fat milk has become so intrenched in the nutritional psyche, it persists despite the absence of evidence,” he says. “To the contrary, the evidence that now exists suggests an adverse effect of reduced-fat milk.”

There are also some hints that the benefits of milk for bone may also be, well, exaggerated. In July 2012, a study published in the Archives of Pediatrics & Adolescent Medicine followed a group of 6,712 girls over seven years to assess who was likely to develop a stress fracture. They found that neither calcium nor dairy intake was associated with a lower risk of stress fractures. Similarly, a 2003 Harvard Nurses’ Health Study that followed over 72,000 postmenopausal women for 18 years found that milk did not seem to lower risk of hip fractures.

Ludwig and Willett aren’t out to tank the dairy industry, but are simply hoping to draw attention to the need to reconsider milk recommendations — this time based on the scientific evidence. They say the data supports a recommendation for drinking up to three cups a day, but doesn’t support the advice that it has to be low-fat or skim milk.