WASHINGTON — More than 20,000 taxpayers have been targeted by fake Internal Revenue Service agents in the largest phone scam the agency has ever seen, the IRS inspector general said Thursday.

Thousands of victims have lost a total of more than $1 million.

As part of the scam, fake IRS agents call taxpayers, claim they owe taxes, and demand payment using a prepaid debit card or a wire transfer. Those who refuse are threatened with arrest, deportation or loss of a business or driver’s license, said J. Russell George, Treasury inspector general for tax administration.

Real IRS agents usually contact people first by mail, George said. And they don’t demand payment by debit card, credit card or wire transfer.

The inspector general’s office started receiving complaints about the scam in August. Immigrants were the primary target early on, the IG’s office said. But the scam has since become more widespread.

Tax scams often escalate during filing season, George said. People have been targeted in nearly every state.

“This is the largest scam of its kind that we have ever seen,” George said in a statement. “The increasing number of people receiving these unsolicited calls from individuals who fraudulently claim to represent the IRS is alarming.”

The script is similar in many calls, leading investigators to believe they are connected. The inspector general’s office is working with major phone carriers to try to track the origins of the calls, the IG’s office said.

The scam has been effective in part because the fake agents mask their caller ID, making it look like the call is coming from the IRS, George said. In some cases, fake agents know the last four digits of Social Security numbers, and follow up with official-looking emails.

They request prepaid debit cards because they are harder to trace than bank cards. Prepaid debit cards are different from bank cards because they are not connected to a bank account. Instead, consumers buy the cards at stores, and use them just like a bank card, until the money runs out or they add more.