A maths professor has forced Tesco into a humiliating climbdown after exposing how the retailer got its sums wrong when calculating interest on its credit card.

Nigel Cutland, emeritus professor of mathematics at the University of York, spotted that Tesco charged him interest on his credit card – which promised 0% for 12 months – after 11 months and a week.

Tesco initially argued that "12 months means 12 monthly statements, not calendar months". But its response prompted the professor, more used to dealing with the precision of stochastic equations, to take his case further.

"My simple point was that they couldn't define 12 months to mean what they like – that only happens in Alice's Wonderland. While a relatively small amount, if this is the way they act for all such situations, they are making quite a bit of gain. I suspect it is a systemic fault."

After what he said was a "stern phone call to someone senior in complaints", Tesco looked at his case again and accepted there had been a "technical error". His interest bill, initially £76.90, was cut to £22.80.

A significant number of Tesco card holders may now be in line for refunds after the supermarket giant said it would run checks on other accounts to identify if there has been overcharging elsewhere.

Tesco has 2.7 million credit card customers, many attracted by its 12-month 0% interest deal. Tesco's financial services arm is one of its most lucrative areas of business, enjoying a profit margin of 27%, compared to 6% on groceries.

In a statement, Tesco said that customers receive the full introductory period. However, it added: "As a precautionary measure, we are running checks to make sure other accounts opened at the same time have not been affected."

Cutland said he is not seeking any personal publicity over the case but is concerned that the general public is left bewildered by APRs and rate calculations.

"The credit card people take advantage – deliberately or inadvertently – of the many people who don't check things like interest calculations, or even don't know how to."

The retailer's credit card holders are also being hit with a little-publicised charge when using their plastic abroad, which could be raking in millions of pounds a year for the supermarket group, according to a Guardian Money investigation.

The latest TV advert for its Clubcard credit card extols the virtues of using its plastic to buy items abroad, but Tesco's banking arm, along with Royal Bank of Scotland and NatWest, has been accused of "manipulating" the exchange rates used to convert overseas purchases into sterling, so that their cardholders pay more than people holding other cards.

Research by the Guardian indicates that in many cases Tesco is imposing an extra mark-up of up to 2% or more, on top of its published 2.75% commission fee for foreign transactions. That mark-up could translate into an extra £4 on a €200 purchase. Tesco said it had set its own rates for foreign transactions since 2009, and that it was entirely transparent.