As a veteran of EU trade negotiations, Nick Clegg always knew that life in Britain's first peacetime coalition government since the 1930s was never going to be easy.

The deputy prime minister may have looked relaxed last night when he pitched up at the Spectator's summer party with David Cameron. Sipping orange juice – no Pol Roger champagne for them in these straitened times – they looked completely at ease as they chatted to the magazine's editor Fraser Nelson.

But Nelson is a leading opponent of electoral reform. Clegg was today given a taste of the formidable power of this group after the Guardian revealed that the Lib Dem leader will announce that a referendum is to be held on 5 May next year on whether to introduce AV.

Clegg wants to hold the referendum on that date to maximise turnout. Elections to the Scottish Parliament and Welsh Assembly will also take place on 5 May.

Lib Dems also fear that a delay beyond 5 May could undermine the chances of a yes vote because the party is bracing itself for a setback in the Scottish and Welsh elections. The elections will take place a month after the most swingeing spending cuts since the second world war are introduced.

But the referendum can only take place after enabling legislation has been passed in Parliament. Under the coalition agreement, David Cameron will whip his MPs to vote for the legislation. Once that is passed the Tories will be free to campaign for a no vote.

Clegg learnt today that he will face opposition from a formidable combination of the Tory 1922 committee and elements of the Labour party. Both sides will face constraints. The Tories accept they are duty bound to support the enabling legislation. Key Labour figures will support the AV referendum because they pledged in their manifesto to hold one if they won.

But this is how the two sides will make Clegg's life very difficult over the next year:

•Opposing the proposed date of the referendum. Bernard Jenkin, the former Tory frontbencher, has been appointed by the 1922 committee to negotiate with ministers. A passionate opponent of AV, he accepts a referendum must be held. But he is adamantly opposed to holding it on 5 May. Jack Straw, the shadow justice secretary, also questioned the date yesterday.

•Raising the threshold. Jenkin will demanding a high threshold, possibly modelled on the system used for the referendum on Scottish devolution in 1979. A narrow majority of voters (51.6%) supported a Scottish parliament but the referendum failed because the rules required that at least 40% of the overall electorate voted yes.

Jenkin tells me:

The Conservative party has been consistently in favour of thresholds. It could be that 40% of the electorate would have to vote yes. That would require an 80% turnout to get a yes vote.

•Opposing the plans, to be announced by Clegg on Tuesday, to shrink parliament by 10% as the size of constituencies are equalised. The Tories support this because it was a key manifesto pledge. But Labour is opposed to it because they believe it is designed to hit Labour because there are 3.5m unregistered voters who are more likely to support Labour.

I picked up an intriguing detail on this tonight. Tory MPs have been told by Downing Street that even if the referendum is passed, the next general election will not be held under AV until the boundary review to shrink the constituencies has been carried. This is designed to prevent the Lib Dems abandoning the coalition the day after a yes vote to trigger a general election.

Clegg, who has been highly critical of Labour in recent weeks, will hope the party's next leader will be more supportive than Straw. Ed Miliband said today he would campaign for a yes vote "whenever the referendum takes place". Ed Balls said he supports AV but is concerned about the timing.

So where does this leave Clegg and Cameron?

•The Lib Dem leader is wisely acting with great caution and waiting to outline his plans to parliament. But with the 1922 committee and an old warhorse like Straw on the loose, Clegg will need to be careful about leaving a vacuum between now and his statement on Tuesday.

•The prime minister will have mixed emotions. He passionately supports Britain's current first past-the-post electoral system and will campaign for a no vote. But the prime minister knows that a no vote will destabilise the Lib Dems and may even be seen as a vote of no confidence in the coalition as a whole.

Is the prime minister secretly praying for a yes vote? Take a look at Benedict Brogan's superb blog today under the headline: Did David Cameron choose May 5 because he secretly wants a 'yes' vote?