Police and passersby at the scene of the Taser incident outside Buckingham Palace. Photograph: John Fahey/PA

A man believed to have been holding a knife to his own throat has been Tasered by police outside Buckingham Palace during the changing of the guard ceremony.

Police said a man was seen with two knives by the centre gate at 11.50am on Sunday. "He was not making threats to members of the public but he was challenged by police. He acted aggressively and a Taser was discharged," a spokesman said.

A man, thought to be in his 50s, was arrested on suspicion of affray and taken to a central London police station.

The Queen and Prince Philip were at Sandringham at the time.

A palace spokeswoman said they would not comment as it was a police matter. Police at the scene told members of the public that a man "tried to make an exhibition of himself during the guard change" and that there were "mental health issues".

The scene was cordoned off with police tape 30 metres from the palace gates and an ambulance was parked to the side of the building.

Inside the cordoned-off area lay two knives with six-inch blades, a pair of Nike high-top trainers, a hat and a black carrier bag containing paper material. There was also Taser wire on the ground.

Kevin Burrows, 33, from Surrey, said he saw a man break through a cordon as the changing of the guard was about to happen.

"I thought 'Oh my God, I can't believe it'. There must have been about 15 police officers on foot who surrounded him once he'd been Tasered," Burrows said. "The police saw him really quickly and were obviously cautious. Everybody was standing back when it happened and people were actually quite calm, I think everyone was surprised. They had to divert the procession away from him. I think he was making his way to the guards."

In a video posted on YouTube a man is seen taking a series of swipes at an officer as the stun gun is fired. But as it connects he falls to the ground watched by hundreds of tourists who had assembled to see the changing of the guard and take photographs of the palace.

Witnesses said the band played on as the chaos unfolded. Mr Burrows added: "The man didn't have the chance to get close to anyone as the police were right on him in seconds."