Increasingly lavish credit card rewards have become a glitzy perk for millions of well-to-do Americans. But there’s an unseemly and little-talked-about open secret behind these rewards: They’re being paid for in part by poorer Americans.

But nothing is free. Banks at least partially finance credit card rewards through a fee on the store that accepts your card. The merchants then turn around and raise prices on shoppers. (These “interchange" fees are just one way that banks earn money—they also profit from charges such as interest, annual fees, and late-payment fees.)

The truth is, everyone pays for a tiny fraction of the ever-increasing rewards pie. But for the millions of poorer Americans who don’t have the credit score and history to obtain a premium card, this situation is gallingly inequitable.

The game isn’t fair. But does that mean you shouldn’t play?

How interchange fees work

Say you buy a $180 bedside lamp. Your transaction routes to the bank that issued your card so that it can verify the payment (for example, Citibank, if you use the Citi® Double Cash). The bank keeps a small percentage of the transaction (generally 1% to 3%) as a fee and then transfers what you paid (minus the fee) to the store’s bank. The store’s bank keeps a fixed fee (let’s say 50¢) and then delivers the remainder of the money to the store. This process can take a few days.

In this example, $180 for the bedside lamp turns into $176.04 after Citi takes its cut (assuming a 2.2% fee), and then $175.54 after the acquirer’s cut. That means the store misses out on $4.46 (or about 2.5%) just to make the sale.

Merchants don’t idly suffer this fee. They proactively and preemptively raise prices in anticipation of taking this hit. That means all shoppers—not just credit card customers—pay higher prices so that some people can have the privilege of swiping their cards and earning rewards.

There isn’t just one interchange fee; a whole raft of them apply to different types of cards and spending. For example, Mastercard charges higher interchange fees for World Elite cards (the designation for the Citi® Double Cash) than for simple World cards. Mastercard also charges one fee for supermarkets and another for restaurants. Big businesses with higher sales have different costs than smaller, mom-and-pop operators.

But here’s the open secret: The better the rewards card, the higher the interchange fee.

Who’s left out in the cold

You might think this is all no big deal. So what if banks and networks earn some money for taking on the risk of operating an instantaneous credit-payment system? Sure, merchants are out a few bucks here and there because of these fees, but they also benefit from the payment system since people are likely to spend more money with their rewards card than with cash. Does a store really want you to think twice about how to pay when you’re checking out?

But there’s one group this system really harms: people without premium reward cards or any credit cards at all.

“[C]onsumers who do not use credit cards may be made worse off by paying higher prices for goods and services, as merchants pass on their increasing card acceptance costs to their customers,” states a 2009 Government Accountability Office report on interchange fees (PDF). And the issue hasn’t gone away since then.

Although the vast majority of Americans own a credit card, only 60% of those earning less than $40,000 a year do. Meanwhile, half of all Americans have a non-prime FICO credit score (less than 720), and you generally need something in the range of 740 to 760 to have a really good shot at qualifying for a top rewards card. The more you earn, the better your credit score tends to be.

This adds up to an uncomfortable but undeniable fact: Poorer Americans are footing a substantial part of the bill for the credit card rewards of wealthier Americans.

Taking advantage of an unfair game

Aaron Klein has thought a lot about banking inequality. He’s a fellow at the Brookings Institution, and was the deputy assistant secretary for economic policy at the Treasury Department from 2009 to 2012. He wrote an essay for NBC last year that laid out banking’s structural inequalities, including credit card rewards. Partway through, he archly notes that he wrote the article while on a flight he paid for with rewards.

“I’m writing about how the system perpetuates income inequality, while realizing that as a person with a high income I am a beneficiary of it,” Klein said in an interview with Wirecutter.

So what can individuals do to address credit card inequality? Well, not much, said Santa Clara University professor of finance Meir Statman (who owns a United Airlines card, the Uber Visa, and the Costco Anywhere Visa®).

“Refraining from award cards is futile as a tool for change,” Statman said in an email. Besides changing laws and regulations, “we can help the poor though contributions to charity organizations (mainly [the National Alliance for Mental Illness] in my case) serving the poor and needy.”

The law may change one day. For instance, the Durbin amendment, part of the larger Dodd-Frank banking reform bill of 2010, lowered interchange fees on debit card transactions. One day such regulations may limit what banks can charge more broadly, which would most likely put a damper on the huge rewards so many people have come to love.

When the European Parliament limited interchange fees in 2015, banks reacted swiftly, notes Mercator Advisory Group’s Brian Riley. In the United Kingdom alone, RBS, Capital, Tesco, and other institutions devalued their rewards.

In the meantime, this is a tricky situation for American shoppers. Merchants increase prices to offset the cost of accepting credit cards, and that helps pay for the “freebie” perks that rewards fans love.

What you do with that information is of course up to you. But you should at least know who’s paying for your perks. It isn’t just the bank—it’s also the American public.

Wirecutter is a list of wonderful things by Brian Lam and friends, founded in 2011 and a part of The New York Times Company since 2016. Have a question? Just ask.