BANK WITHDRAWAL: Goldman Sachs' 43-story HQ will soon open -- but the bank led by CEO Lloyd Blankfein (inset) is complaining that there won't be enough anti-terror cops to protect it, since the city and state are scaling back. Photo: Brian Branch Price

Goldman Sachs, the wealthiest bank on Wall Street, is locked in a bitter battle with city and state officials over the number of cops available to patrol its gleaming new world headquarters at Ground Zero.

The bank is close to completing its $2.4 billion, 43-story skyscraper on Vesey Street in Battery Park City and says it will be ready to move in by the end of this year.

But the big move could be delayed as the city and state scale back an expensive security agreement with the bank. The squabble is dealing yet another blow to the redevelopment of Ground Zero — and possibly putting taxpayers on the hook for $320 million if the Goldman-government accord collapses.

In 2005, the state and city struck a deal with Goldman to provide the bank with $1.65 billion in tax-free Liberty Bonds, which have saved the bank at least $9 million a year in interest payments. The deal included $115 million in other tax breaks and cash grants.

One key incentive was the promise to provide Goldman with a much higher level of security than usually given to a Wall Street firm. Under the terms of the contract, the city pledged manpower, and the state was in charge of security infrastructure such as blockades, surveillance equipment and guard stations.

But now the city has told Goldman it will scale back the number of cops — pointing to budget woes and arguing that the higher levels are not yet necessary, given the long delays in opening office buildings at the World Trade Center site.

Sources also said the state is so far behind in its infrastructure plans that the NYPD would be unable to use the number of cops outlined in the deal, even if it had them to spare.

Now, Goldman is fuming over the security scale-back, sources said.

It even threatened to go public with what it sees as breach of contract that could leave the new tower vulnerable to attacks, sources said.

But city and state officials haven’t budged, banking on the notion that a whining Wall Street firm at the center of the city’s economic meltdown would not garner public support.

“They think that the bank will be a laughingstock if it complains about not getting enough free security, especially after Goldman has said it will pay out millions of dollars of [executive] bonuses this year despite the bad economy,” a source close to the security talks said.

Just this summer, Goldman reported record-setting profits — and CEO Lloyd Blankfein had to urge execs to curb their spending habits as the bank paid back $10 billion in federal bailout money.

But a Goldman source countered, “The issue is that the redevelopment of that whole area calls for a combined city and state security plan.

“The city and the state have an obligation to everyone who lives and works downtown and that includes us. We have the right to appropriate levels of security.”

The dispute could be a costly problem for taxpayers.

As part of the deal, Goldman would be in line for a $320 million payout if the building is not ready by 2010 because of a failure to complete the security plan or road projects.

The Goldman source insisted the bank has no interest in claiming the cash. It just wants the security upgrade as well as the promised transportation links in place by December, the source said.

“We have a big problem with the state over the fact that the transportation issues are not resolved,” the source said, adding, “On balance, the city has always been very open with us, but the state we have real problems with.”

Goldman Sachs, which is currently based at 85 Broad St. but has several other offices downtown, is slated to start moving into the new tower in January.

The NYPD has boosted security downtown to massive levels since the 9/11 attacks. Heavily armed cops patrol Wall Street while a so-called “ring of steel” is implemented around the Financial District.

“The city has met all of its obligations under the agreement and we believe Goldman Sachs is satisfied with our performance,” said Jason Post, a City Hall spokesman. A state spokesman would only say that “good faith” discussions continue.

Sources said additional police manpower — which the city decided in 2006 to add on its own — is now being scaled back because of the economy and the fact that the new World Trade Center complex is behind schedule.