Pro-Russia gunmen in an administration building in Luhansk, eastern Ukraine. Photograph: Alexander Zemlianichenko/AP

Ukraine's acting president has admitted his government has practically lost control of the east of the country, with his security forces "helpless" to stop a rolling takeover by pro-Russia gunmen.

Oleksandr Turchynov said numerous Ukrainian military and security personnel had defected to the rebels, taking their arms with them. Using the language of defeat, he told a meeting of regional governors: "I will be frank. Today, security forces are unable to take the situation in the Donetsk and Luhansk regions quickly under control."

His comments came after anti-Kiev militants seized a string of official buildings in the eastern city of Luhansk on Tuesday. Only the city's police station remained under nominal central command.

Standing outside Luhansk's police headquarters on Wednesday, Tatiana Pogukay was close to tears. "We are still here. We didn't surrender. But Kiev betrayed us," she said. Her emotion was understandable. The previous night she and her colleagues had fended off an armed attack by 50 pro-Russia separatists. The gunmen in fatigues attempted to storm the building. One fired warning shots in the air; some scaled the roof; others rammed the rear entrance with a Kamaz truck.

The officers inside managed to stand their ground. They tossed smoke and stun grenades at their assailants from upper-floor windows. A frightened group of conscripts was allowed to leave. Eventually the separatists called it a night – with the police still in control of their four-storey regional HQ, at least for now. "We didn't give up our weapons. We defended this place with honour," Pogukay, a police colonel, said.

A woman gives food to an armed man in front of the occupied regional administration building in Luhansk. Photograph: Zurab Kurtsikidze/EPA

Despite some heroic resistance, the reality is this: most of east Ukraine is now a separatist fiefdom. The balance of power is tipping. It is moving inexorably away from the beleaguered government in Kiev, and towards militants supported by Moscow who are on the brink of declaring their own autonomous state.

With its Khrushchev-era apartment blocks and neo-classical Stalinist buildings, Luhansk – population 445,000 – already looks like Russia. The Russian border is 20 miles (35km) away. In a matter of weeks Luhansk will either be Russia, or a political entity allied with it.

On Wednesday the city's volunteer-soldiers were taking it easy after a remarkable 24 hours in which they seized municipal power. In early April a militia led by Soviet Afghan war veterans occupied Luhansk's security service agency HQ, in the centre of town. They helped themselves to its formidable arsenal: shiny new Kalashnikovs, pistols, grenades, and rocket launchers. Last week a new Moscow-approved "people's governor", Valery Bolotov, appeared. Bolotov sent Kiev a list of "demands". He set a deadline, which expired on Monday.

After failing to get a reply, the rebels carried out an effortless coup a day later. First, a 3,000-strong crowd encircled Luhansk's regional administration building, which overlooks a pleasant park and a statue of the moustached poet Taras Shevchenko. A couple of youths in masks smashed in the windows with crowbars; then the masses poured in. Riot police trapped in a courtyard did nothing. Miserable and humiliated, they eventually left.

On Wednesday masked armed volunteers were standing guard outside their new HQ; sandbags had been piled up before ground-floor windows; a nascent tyre-wall was taking shape on the pavement. One militiaman, Alexander, was happy to chat. He said despite the change in management it was business as usual, with council staff working as before. "Our job is to check IDs," he said. A wellwisher gave him a carton of Bond Street cigarettes. Nearby, a group of women were discussing the dramatic events of the previous day.

One old lady in a headscarf seemed confused by this rapid transfer of control. She asked Alexander if he had come from the Maidan, Kiev's pro-western protest movement?

"We're against fascism," Alexander answered.

She remained confused. Another woman intervened helpfully and said: "Don't worry, dear. Everything will be like in the Soviet Union again. We will have our Victory Day on 9 May. It will all be OK."

The old woman's face brightened. She grasped Alexander by the hand, and before walking down the street told him: "History will remember you."

Alexander, his face hidden by a balaclava, declined to give his surname. But he said he was a 39-year-old miner with a family. A "referendum" on Luhansk's future status would take place on 11 May, he explained. After that, he said, the region would probably join Russia, though he wasn't certain. Why had he joined "Luhansk's people's militia"? "There's been a fascist takeover in Kiev," he replied. He added: "Victory Day is sacred for us. I've heard the Kiev government wants to replace it with a gay pride parade."

The insurgents now occupy practically all of Luhansk's official buildings. Armed men on Wednesday patrolled outside Luhansk's city hall, also taken on Tuesday; one had a Kalashnikov with its grip decorated in Russian red, white and blue. Others peered out from inside the procurator's office down the road; they emerged to sweep away broken glass from their break-in the night before. A new Russian tricolour flew above the regional appellate court. The militia also dropped into the TV station for a chat.

In other parts of the east it was the same story. In the town of Gorlovka, 20 miles from the regional capital of Donetsk, militants hijacked the city hall early on Wednesday, as well as a second police station. In Donetsk others grabbed the tax and customs office. Each occupation follows the same revolutionary template: gunmen, who do the heavy lifting; takeover; tyre barricades; call for a referendum. Ukraine still has tanks and troops in pockets in the east. But the reality is that Kiev's authority has vanished, probably forever.

Instead of trying to wrest back control of Luhansk and Donetsk, Turchynov said on Wednesday his forces would concentrate on defending the provinces of Kharkiv, in the east, and Odessa, in the west. This will be difficult. The Kremlin's plan appears to be to resurrect the historic region of "New Russia", a large Russophone chunk of southern and eastern Ukraine.

At Luhansk's police station, meanwhile, Pogukay said she was incensed with Turchynov after he called the east's law enforcement officers "traitors". "First Viktor Yanukovych [Ukraine's ex-president] betrayed us. Now Turchynov betrayed us," she said, her voice shaking with feeling. "This is on his conscience." On Tuesday, just before the militia tried to seize the station, the regional police chief, Vladimir Guslavsky, faxed his resignation to Kiev, she said. He didn't get a reply. "He's a man of honour. He's staying in his post until they send a replacement," she said.

During the raid the insurgents managed to claw off half of a Ukrainian trident sign on the station's front wall. They also raised a Russian flag on a low overhanging roof; someone took it down again later. In anticipation of further attack, the police had sandbagged the entrance and ground-floor windows. Pot plants were visible in several upper windows; the lights were on. Ukraine's deputy interior minister had phoned up to offer his support, Pogukay said. But no one else from Kiev had bothered to pick up the phone. Surely the situation was now hopeless for the police? "We're not traitors like Turchynov said," she replied. "We'll fight to our last breath."