There’s no natural reason for some wheels of cheese to be bright orange, but most of us don’t think about it all that much. Sure, we know the fluorescent shade of boxed mac and cheese is food coloring fakery, but we don’t think of pale orange cheddars or goudas as looking unnatural.

Of course, no cheese is naturally orange. Milk does not have orange pigment in it, and none of the bacterial processes going on inside aging dairy turn it such a bright hue.

If you love an intriguing origin story, this version of how cheese became orange is for you:

Back in ye olde 17th century (or possibly 16th, no one seems sure), English dairy farmers were looking to make a few extra shillings, so they started skimming the fat off the milk used to make their cheese. They could turn this fat into other products, like butter, or just sell it straight to consumers. The trouble was that the cheese made from low-fat milk didn’t have the same creamy yellow hue to it, and was thus identifiable as lower quality cheese. Not to be thwarted by a few fat globules, the farmers simply started adding coloring to mimic the hue of a full-fat product.

Over time, added coloring became a way to make your cheese stand out, and eventually certain products became associated with particular orange or yellow hues. And the tradition continues today.

If you like origin stories with more plausibility but less dastardly intrigue, there’s another version. Cheese made from cow’s milk in the spring and summer has a more buttery hue to it because the herd is out grazing on grass rather than chomping on feed. Applying a colorant helped farmers even out the appearance of their cheeses over the course of the year. It was all pretty much the same quality, so farmers were just assuaging customers’ concerns over the varying hue.

Pick whichever story you prefer, because it’s not clear which one is more accurate. But we do know that two of the core facts are true: 1. That the color of cheese used to vary over the course of a year, and 2. That over time some cheeses have become traditionally orange, even though no cheese is naturally that color.

“Today it’s used to bring out the tradition of the cheese, more so than to even out fluctuations over the year,” says Gina Mode, one of the lucky few people who gets to work with and research cheese at the Center for Dairy Research at the University of Wisconsin-Madison (she also grew up on a dairy farm). She explains that cows today, and really for much of the past century1, are fed year round on grain-based feed, not grass. And it’s the grassfed cows that have variations in their milk.

Grass, unlike grain, contains lots of beta carotene, an orange pigment that’s also found in carrots, pumpkin, and sweet potato. Cows turn some of their dietary beta carotene into vitamin A, and the rest gets stored in their fat cells. That same fat also makes its way into the milk. Since the beta carotene is bound up inside the fat globules, though, the milk itself isn’t all that yellow. It’s only when you add enzymes in order to make cheese or butter that the globules break down, revealing the pigment.

This means that cheese from grassfed cows has a significantly yellower hue to it. Just take a look at the variation, even in uncolored cheddars.

Goats’ milk cheddar is the whitest, because goats convert all of their dietary beta carotene into vitamin A and leave none behind to color their cheese. Then they get increasingly yellow and top out with the New Zealand cheddars because, as Mode explains, “New Zealand has much more grass-fed [cows], so they have a richer, golden color to their cheese.”

There’s also variation based on the type of cow producing the milk. “Some cows just pass the beta carotene right through,” Mode says. “The most famous of those is the Guernsey. There used to be a Golden Guernsey brand that was famous because it had this beautiful golden color.” It was for that reason that Guernseys were prized for making butter, the color of which also heavily depends on the beta carotene content of the milk it comes from.

Butter also used to be a richer yellow color, and many manufacturers used to add a pigment to even the hue over the course of the year. This is why margarine was so aggressively yellow—to make it look like the finest butter. Farmers pushed back on this so much that state legislatures started enacting laws to keep margarine from looking too much like a real dairy product. “I’m too young to remember this, but my parents talk about how it used to be illegal to color your margarine because Wisconsin was so pro-dairy,” says Mode. “If you bought margarine, it was completely white and it had a packet of annatto in there for you to stir in yourself. People used to go across the border just to buy yellow margarine.”

Those restrictions were eventually lifted, and today most margarine is back to an almost unnatural yellow color. But that fact that people cared enough that their non-dairy spread looked like a dairy product illustrates why we bother to color food in the first place: it affects the taste.

Not literally, of course, because annatto is tasteless. It’s a fat-soluble pigment that covers the seeds of the Bixa orellana L. plant, and in the tiny quantities used for cheesemaking it changes nothing about the true flavor. (Incidentally, you can add beta carotene directly to the milk, but the University of Guelph’s Food Science school notes that it is “too yellow and makes the cheese taste like carrots”).

Still, our perception of a food affects how we think it tastes, and thus orange cheeses and yellow butters seem different than pale products.

All cheddar cheese, for example, is actually white (barring the grassfed stuff), but some people have a preference for the orange stuff because it seems to taste better. Blindfolded, they’d never be able to discern the difference. The orange is just a pigment—but it works.

The use of annatto in coloring cheese may have started with cheddar. Mode says that one of the first to start using orange coloring for marketing purposes was an English variety called Red Leicester.

Red Leicester is a hard cheese similar to cheddar that uses annatto to produce a dark orange hue. When the farmers selling Red Leicester began their campaign, cheddars were all white, so their cheese’s orange tint made it stand out.

Other cheesemakers followed suit, and over time certain varieties acquired a tradition of coloring. Some are the deep orange of Red Leicester, others a lighter orange, and still more a light buttery color. Consumers now expect gouda, edam, and havarti to have creamy hues, so most manufacturers add a tiny bit of annatto to produce that rich color (by the way, you’ve been saying “gouda” all wrong. It’s GOW-duh, not GOO-duh). One strange exception: Muenster isn’t trying to mimic a beta carotene color at all. The type made in England has an orange exterior that comes from the microorganisms living on the surface, but the variety made in the U.S. uses annatto to merely mimic that pattern.

Ironically, it’s now the natural grass-fed hue that makes cheese stand out in the U.S. “If you’re an artisan, it’s one of those things that’s unique and special for your cheese,” explains Mode. “Like Uplands’ Pleasant Ridge Reserve, that’s a famous one where you can actually taste a difference from the milk as well.” Uplands only makes that cheese from May through October because the quality of the milk is so distinct. In the fall, when the cows feed on hay instead of fresh grass, they make another type of cheese that doesn’t rely on beta carotene for its flavor or color. And you really can tell the difference.