KIEV, Ukraine — Ukraine's government said Wednesday it will pull its troops from Crimea, where Russian troops and Crimean allies are seizing military bases and officers.

Masked armed men assumed to be Russian military seized Ukraine's naval headquarters in the city of Sevastopol on the Black Sea in Crimea and took away a Ukrainian admiral.

Ukraine's defense minister and deputy prime minister put off a trip to Crimea in what they said was a bid to avert an escalation in hostilities after the self-appointed prime minister in Crimea, Sergei Aksyonov, warned that they "are not welcome," Interfax news agency said.

The hostilities follow a decree by Russian President Vladimir Putin that the former captive of the Soviet Union, Russia's predecessor, is now part of Russia despite warnings from the White House and Europe that Moscow will pay "costs" for a takeover.

U.S. Vice President Biden, in the former Soviet republic of Lithuania, declared that the U.S. was "absolutely committed" to defending its allies.

"Russia cannot escape the fact that the world is changing and rejecting outright their behavior," Biden said, after meeting in Vilnius with Lithuanian President Dalia Grybauskaite and Latvian President Andris Berzins.

Ukrainian soldiers in the province of Crimea had been left in limbo after the Crimeans voted Sunday to join Russia, a referendum that Ukraine and the West said was unconstitutional. They held onto their bases, which were surrounded by Russian troops and mobs of pro-Russian Crimeans.

A confrontation at one base on Tuesday that left one Ukraine servicemember dead prompted an order from Kiev that the Ukrainians could fire their weapons to protect themselves. That did not happen though in Sevastopol.

Interfax, a Russian news agency, said "unknown" troops took over the base and Ukraine's navy commander, Serhiy Hayduk, was arrested and the Russian flag raised over the base.

Russia said Ukraine's government is no longer legitimate because it ousted president Viktor Yanukovych, a Moscow ally, for the shooting of more than 80 protesters in Kiev.

Ukraine said Wednesday it will exit the Russia-dominated Commonwealth of Independent States set up after the collapse of the Soviet Union, the country's security chief said Wednesday. Andriy Parubiy, secretary of the National Security and Defense Council, said Ukraine would hold military exercises going forward with the United States and Britain.

Russians who want to visit Ukraine will be required to get a visa, the ITAR-Tass news agency reported. Citizens of the two countries had been able to travel back and forth without visas since 1991, when Ukraine became an independent country.

Many Ukrainians fear that Russia could also now invade eastern Ukraine in response to parliament's installation of a pro-Western government. Putin's spokesman Dmitry Peskov told the BBC that Russia would not do so, but some here say he may first blackmail the east by hiking the price of energy exports.

"I think that Russia will fight economically outside Crimea," student Dmitri Alexandrovich said in Kharkiv, a Russian-speaking city in eastern Ukraine. '"Then, once the country is on its knees, 1,000 parts will break away and ask to join Russia."

Contributing: Luigi Serenelli; Associated Press

Russia's President Vladimir Putin addresses a joint session of Russian parliament on Crimea in the Kremlin in Moscow on March 18. Putin pushed every emotional button of the collective Russian psyche as he justified the incorporation of Crimea. Alexei Nikolsky, AFP/Getty Images

A woman watches TV showing Putin speaking about the situation in Ukraine in Moscow, Russia, on March 4. Vladimir Putin met with journalists at his residence outside Moscow to explain his position on the Ukrainian events, saying that he sees no need to send Russian troops to Ukraine now. Yuri Kochetkoc, EPA

Putin, center, shakes hands with the commander of the Western Military District Anatoly Sidorov, right, upon his arrival to watch military exercises near St. Petersburg, Russia, on March 3. Mikhail Klimentyev, AP

Flanked by Russian Defense Minister Sergei Shoigu, left, and the commander of the Western Military District Anatoly Sidorov, Putin arrives March 3 to watch military exercise near St.Petersburg, Russia. Mikhail Klimentyev, AP

Putin attends a wreath laying ceremony at the Tomb of the Unknown Soldier by the Kremlin wall to mark the Defenders of the Fatherland Day holiday in Moscow on Feb. 23. Alexei Druzhinin, AFP/Getty Images