Key legislators in Albany and New York City yesterday flatly rejected extending Mayor Giuliani’s term – even as members of the mayor’s inner circle split over whether he should try to remain in office past Dec. 31.

“I don’t contemplate any circumstance that would get the Senate back to change the election process in New York state,” declared state Senate Majority Leader Joseph Bruno, the mayor’s most reliable GOP ally in Albany.

“It’s too late,” Bruno added.

A few hours later, City Council Speaker Peter Vallone – a Democrat with close ties to the mayor – said he’s also in no mood to tamper with term limits.

“It’s over,” he said. “We’ve been through that fight.”

Some in the mayor’s inner circle, including Deputy Mayors Joseph Lhota and Bob Harding, have told Giuliani he can’t leave office at this critical juncture – despite the law that bars any city official from serving more than two consecutive terms.

Sources said Judith Nathan, the mayor’s girlfriend, shares that view.

The adulation accorded Giuliani for his masterful leadership after the terrorist attack on the World Trade Center provides the mayor with face-to-face evidence every day of the public’s sentiment, said one insider.

“I’ve never seen anything like this,” the source said. “I don’t think he walks 10 steps before someone says to him, ‘You have to stay on.’ He’s hearing the concern and panic in people’s voice about the transition. I think it’s had an impact.”

But other close advisers are worried that any move to tamper with term limits so close to the Nov. 6 election will be viewed as an unpopular power grab.

“Some people are saying it’ll hurt him,” said a second insider.

“He’s at the height of his popularity. There are other ways he can help the city without touching term limits.”

Giuliani dismissed reports that he planned to make an announcement about his future yesterday – saying he had made no decision and wasn’t thinking about politics in the wake of the World Trade Center tragedy.

“It’s a very important decision. I need time to think about it. I need time to talk to people about it,” he said at a news briefing yesterday, barely 24 hours before the polls opened for the delayed primaries for mayor and other city offices.

Giuliani echoed that opinion later in an appearance on “The Late Show” with David Letterman on CBS – but did say the possibility of another term was viable.

“It can be done. It’s really a question of whether it should be,” he said.

At the earlier briefing, the mayor called on voters to do their civic duty – but his tone was hardly enthusiastic.

“I would urge them to vote if they want to,” Giuliani said. “People decide on their own if they want to vote or they don’t want to vote.”

The mayor said it made no sense to write in his name today, since he’s not an eligible candidate .

Mayoral advisers concede there’s no way around the need for legislative action to extend Giuliani’s term.

But one insider wondered what would happen in November if both the City Council and the state Legislature refused to change term limits, yet voters still elected Giuliani.

“If there is a write-in and he wins, what happens? Whoever is the next mayor is going to have a hard time governing,” ^^^^^^^^^^^^^^^^^^^^^^^^^said the mayoral ally.