See more in-depth Ukraine coverage
Interior Minister Arsen Avakov pledged a "very tough response" to the seizure while local media reported special forces had been dispatched to the area.
Russia has built up forces in areas near the Ukrainian border, and the protesters allege that the authorities who took over are nationalists and "fascists" who aim to suppress the ethnic Russians in Ukraine, while the Kyiv authorities have accused Russia of fomenting the unrest as a pretext for sending in troops.
Despite criticism from Russia and Russian-speaking Ukrainians in the East, the protest movement that led to the downfall of Yanukovych was a broadly-based movement that extended across Ukrainian society, and the current interim government, in an attempt to allay concerns about its relations with its pro-Russian constituencies, has said it wants to grant more autonomy to Ukrainian provinces. But Russia has proposed a more centrally devolved federal system for Ukraine, while Ukraine's government sees it as undue external interference in a sovereign country.
In response to the unrest in the country's east, Ukraine's acting President Oleksandr Turchynov called an emergency meeting of Kyiv's national security council.
Ukraine's acting Foreign Minister, Andriy Deshchytsia, said he had spoken by telephone with Russian Foreign Minister Sergey Lavrov and demanded Moscow stop what he called "provocative actions" by its agents in eastern Ukraine.
Lavrov, in a statement issued by his ministry, said there were no Russian agents in the region and that it would be "unacceptable" if Ukrainian authorities were to order the storming of the buildings.
On Saturday the U.S. said it is "very concerned" by violence in eastern Ukraine, which the White House said was happening "apparently with support from Russia." It urged Russian President Vladimir Putin to stop efforts to destabilize Ukraine.
The White House also said Vice President Biden would travel to Kyiv on April 22 to meet with government officials and other groups, and to highlight U.S. support for "a united, democratic Ukraine that makes its own choices about its future path."
Lavrov on Friday warned the Ukrainian government against using force against protesters, saying that such action would derail the talks on settling the crisis between the United States, the European Union, Russia and Ukraine set for next week, as well as any other diplomatic efforts.
It lashed out at the U.S. warning to slap more sanctions on Russia in case of an escalation of the conflict, saying that "an escalation is only and exclusively possible if Kyev dares to do so, relying on massive support of the U.S. and the EU."
Al Jazeera and wire services