Just when you thought figuring out the gas pricing puzzle couldn’t get any more complex, along comes another piece of that puzzle: credit card fees.

According to a report in PRNewswire a new study indicates that drivers are paying up to 10 cents per gallon in credit card or bank fees every time they pump a gallon of gasoline.

What’s worse is that the cost of the bank “swipe” fee is incorporated into the price of the gas regardless of whether or not you pay cash or credit for your purchase. The reason is that the credit card companies require the costs to be borne by everyone.

The fees are set by Visa (NYSE:V) and MasterCard (NYSE:MA), and now represent the second largest operating expense at gas stations, exceeded only by labor costs. Gas stations and convenience stores within those stations paid in excess of $11 billion in fees last year. Because the prices are fixed in secret, they can be increased at any time and in any amount.

Indeed, the prices charged are not necessarily correlated with the increase, or occasional decrease, in the price of gas. While gas prices increased 80% between 2004-2011, credit card fees rose 180 percent over the same period. And don’t think the rise in fees represents fundamental changes to the way the transactions are carried out or processed. The fees are just additional revenue to the credit and debit card companies, and added expense, passed on to consumers, to the confusing pricing of gas.

“These fees have come to be a tremendous burden on convenience stories, most of which are run by small business people,” said Lyle Beckwith, senior vice president of government relations for the National Association of Convenience Stores. “In many cases, the banks are profiting more from the sale of gas than our members.”

With the busiest travel period of the year coming up, and the usual rise in gas prices sure to follow, trying to figure out a gasoline budget just got even harder.