Former Manchester United turned Sky Sports star Gary Neville has vowed not to pull any punches when criticising the performance of Premier League footballers – but said he will never attack fellow television pundits.

Neville, whose colourful broadcasting has won over some of the most tribal football fans, said he would not "con" his audience by holding back in his critiques of Premier League players.

"You can’t con football fans. If I was sat there continuously praising people that I knew, I wouldn’t last five minutes," he said at the Guardian Edinburgh International Television festival on Thursday.

"I don’t speak to a great deal of football players in the country. I don’t speak to agents. I don’t speak to managers. I don’t get friendly with people. Once you get into that world of being influenced by people continuously you then lose your ability to be able to speak."

Neville, who won a string of league and cup accolades in 400 appearances for Manchester United, said he would not be silenced by the unspoken rule of top-flight football and would not shy away from criticising his former colleagues.

"The idea that they can play in the Premier League and not feel that they’re going to be judged is absolutely not going to happen," he said. "The way I like to do it is with a degree of honesty and to try be constructive.

"There’ve been two or three times where I’ve said things and I’ve cringed and afterwards thought I went too far. I said something the other week about a player and thought, ‘that’s wrong, I shouldn’t have said that’. It’s so fast TV, you can’t always come out with the right word to describe the situation.

"Sometimes you come out with the word ‘unforgivable’. Well, why’ve you said it’s unforgivable? He’s only given a goal away, we’re not going to put him in prison for it, are we?"

But asked for an opinion on his fellow pundits, the former England player was reticent. "From my point of view I admire all the pundits," he said.

"I would never ever criticise or comment on what another pundit has said. It’s just a difference of opinion and you have to have a great deal of respect for other people who are trying to do the same thing as you."

As a player, Neville appeared to thrive on being the target of abuse from the terraces. But on television, he said, the "tribal" nature of initial criticism from Twitter users came as a shock.

"I reckon now the pundits get as much criticism as the players. It’s incredible how everyone has a view on pundits," he said.

"I’ve always been judged in what I’ve done, but I didn’t expect it to be as tribal. Social media doesn’t help. When I first started on Twitter I was getting 50-50 positive to negative. Now I’d say it’s probably 90-10."

Neville described his Sky Sports debut in 2011 as "the most nerve-racking experience I’ve had in my life – along with my driving test" and said he gets irritated when asked for his match predictions. "I’m not a betting shop!" he said.

• To contact the MediaGuardian news desk email editor@mediaguardian.com or phone 020 3353 3857. For all other inquiries please call the main Guardian switchboard on 020 3353 2000. If you are writing a comment for publication, please mark clearly "for publication".

• To get the latest media news to your desktop or mobile, follow MediaGuardian on Twitter and Facebook