Members of the punk group Pussy Riot, including Nadezhda Tolokonnikova in the blue balaclava and Maria Alekhina in the pink balaclava, are attacked by Cossack militia in Sochi, Russia. Photograph: Morry Gash/AP

Cossack militia members attacked the Pussy Riot punk group with whips and teargas in the Winter Olympics host city of Sochi on Wednesday.

Members of the group, including Nadezhda Tolokonnikova and Maria Alekhina, who were released from jail in December, were set upon by uniformed Cossacks – who are acting as an informal police force during the games – and plainclothes security men, as they attempted to perform under a sign advertising the Olympics.

The group arrived in Sochi on Sunday and say they have been detained several times since. On Tuesday, they spent several hours in police detention, apparently on suspicion of stealing a woman's handbag from their hotel. After several hours of questioning, they were released.

On Wednesday, the group of five women and one man gathered about 20 miles from where the Winter Olympics are being held. The Cossacks and other security officials moved in. One Cossack appeared to use pepper spray. Another whipped several group members while other Cossacks ripped off their masks and threw the group's guitar away.

"Cossacks attacked Pussy Riot by a Sochi 2014 banner, hitting us with whips and spraying us with teargas, as we were singing the song Putin Will Teach You to Love the Motherland," wrote Tolokonnikova on Twitter.

An official attacks unmasked Pussy Riot members Nadezhda Tolokonnikova (centre) and Maria Alekhina, left during a protest performance in Sochi. Photograph: Morry Gash/AP

The incident lasted only a few minutes and one Pussy Riot member was left with blood on his face, saying he had been pushed to the ground. Police arrived but did not make any arrests.

Krasnodar region governor Alexander Tkachev promised to conduct a "thorough probe" into the incident and prosecute the attackers. Tkachev said in comments carried by the Interfax news agency that the views of Pussy Riot "are not supported by the majority of people in the region" but stressed the importance of abiding the law.

Later on Wednesday, Tolokonnikova, Alekhina and two others held another surprise mini-performance in central Sochi, this time next to Olympic rings in front of the City Hall.

Jumping up and down, one playing a plastic guitar, they sang-shouted in Russian: "Putin will teach you how to love the motherland!" A person dressed as one of the Olympic mascots joined them for a moment in an apparent joke.

Police were watching the performancebut did not intervene. Tolokonnikova and Alyokhina were released as part of a government amnesty in December after serving the majority of two-year sentences for performing a "punk prayer" in Moscow's main cathedral.