After the retirement as Party President of the Workers' Party by Tomás Mac Giolla in 1988, Rabbitte was seen as one of those who wanted to move the party away from its hard left position and alignment with the Soviet Union and international communist and workers' parties. He and some others who had come from the USI via the trade union movement were seen as wanting to bring the party more to the centre.[4] This led Rabbitte, Eamon Gilmore and others to earn the nickname "The Student Princes". In 1992 Rabbitte played a prominent role with Proinsias De Rossa in an attempt to jettison some of its more hard left positions. This eventually split the party.[5]

In 1994 a new Rainbow Coalition government of Fine Gael, the Labour Party and Democratic Left came to power. Rabbitte was a member of the junior ministerial team, serving as Minister of State to the Government, as well as Minister for State at the Department of Enterprise and Employment with responsibility for Commerce, Science and Technology. During his tenure as a junior minister Rabbitte was involved in establishing an anti-drugs strategy as well as enacting legislation which gave the credit union movement more authority. Rabbitte also decided to locate the proposed new state-backed Technology Campus for West Dublin, based on high speed telecommunication links, at CityWest Business Campus beside the N7 motorway, near Clondalkin.

Following the 1997 general election the Rainbow Coalition lost power. The following year saw a merger between the Labour Party and Democratic Left, with Rabbitte participating in the negotiations. In October 2002 Rabbitte succeeded Ruairi Quinn as the new leader of the Labour Party. Under his leadership the party made some gains in the local elections of 2004. Rabbitte has been described as an extremely good performer in the Dáil, often outshining his Fine Gael counterpart Enda Kenny. He is also noted for his anti-Fianna Fáil rhetoric.

Under Rabbitte the Labour Party agreed to enter a pre-election pact with Fine Gael in an attempt to offer the electorate an alternative coalition government at the 2007 general election, which took place in May 2007. This was commonly known as 'The Mullingar Accord' and the proposed alternative government was called 'The Alliance for change'. The Green Party were also anticipated to be likely members of the coalition government on the basis of agreed points of view on many issues covered by the Mullingar Accord.

This move is rumoured to have caused some tension in the parliamentary party, as some members would prefer not to be aligned with any party in advance of an election. The election result did not return a sufficient number of seats for the Alliance for Change to occupy government, even with the support of the Green Party. Rabbitte himself commented on the election result: "This leaves Mr. Ahern in the driving seat". Negotiations between Fianna Fáil and the Green Party resulted in a government being formed on 13 June 2007.

Following the disappointing result in the election for Labour, Rabbitte announced he was stepping down as leader on 23 August 2007. He said in his resignation statement.[2][6] that he took responsibility for the outcome of the recent general election, in which his party failed to gain new seats and failed to replace the outgoing government. He was succeeded as party leader by Eamon Gilmore.

Rabbitte was re-elected on the first count in the 2011 general election. His running mate Eamonn Maloney was also elected. On 9 March 2011, he was appointed as Minister for Communications, Energy and Natural Resources.

Rabbitte was appointed Minister for Communications, Energy and Natural Resources after the 2011 general election.

The journalist Fintan O'Toole, writing in The Irish Times in August 2011, criticised the lack of royalties system for gas and oil in Ireland. He claimed that "The State is about to sign away almost all our resources on terms by far the worst in the developed world".[9] Rabbitte responded with a letter, claiming that the article was inaccurate.[10] Rabbitte's response was in turn criticised by Cian O'Callaghan, a Labour Party member of Fingal County Council, as "misguided".[11]

In July 2014 Rabbitte was replaced by Alex White as part of a reshuffle of the cabinet.