Chandy ventured into the political arena as an activist of Kerala Students Union (KSU), the student wing of the party. He was the unit president of KSU at St George High School, Puthupally, and went on to become the State President of the organization. Chandy completed his college education from CMS College, Kottayam, SB College, Changanassery. Later, he took a bachelor's degree in law (LLB) from Government Law College, Ernakulam.

Chandy started his political career through the Kerala Students Union (KSU), which he served as president from 1967 to 1969. He was elected as the president of the State Youth Congress in 1970.

He has represented the Puthuppally Constituency for decades, having been elected to the Kerala Legislative Assembly in 1970, 1977, 1980, 1982, 1987, 1991, 1996, 2001, 2006, 2011 and 2016. During his legislative career he had also served as Chairman of the Public Accounts Committee during 1996–98.

Chandy has been sworn in as a minister in Kerala four times. He was the Minister for Labour from 11 April 1977 to 25 April 1977 in the first K. Karunakaran Ministry and continued holding the same portfolio in the succeeding first A.K. Antony Ministry till 27 October 1978. He was in the charge of Home portfolio in the second K. Karunakaran Ministry from 28 December 1981 to 17 March 1982. Again, he was sworn in as a minister in the fourth K.Karunakaran Ministry on 2 July 1991. He was in charge of Finance Portfolio and resigned from the cabinet on 22 June 1994 due to difference with the Chief Minister.

The results of the parliamentary elections in May 2004 saw the Indian National Congress not winning a single seat in Kerala. The sitting Chief Minister, A.K. Antony, was forced to resign and accept responsibility for the poor results. On 30 August 2004, Oommen was elected the Congress Legislature Party leader at the end of a meeting by AICC observers and clearance by the Congress president, Sonia Gandhi. In what may be seen as a reflection of his work as Chief Minister, the Congress-led alliance was defeated, but managed to retain 40 out of 140 seats in the assembly and boost its vote-share by nearly 10% after the general election rout. He resigned as Chief Minister on 12 May 2006 following the defeat of his party in 2006 Assembly Elections.

Oommen Chandy was the leader of opposition in the twelfth Kerala Legislative Assembly. Under his leadership UDF marked victories in Lok Sabha Election 2009, gaining 16 out of 20 parliament constituencies in Kerala, and Local Body Election 2010. In the history of Kerala politics it's the first time that the UDF got an upper hand in local body elections.

After winning the closely contested 2011 assembly election, Congress legislative party unanimously elected Oommen Chandy as its leader. At the Congress Legislature Party meeting, Chandy's name was proposed by Kerala Pradesh Congress Committee (KPCC) president Ramesh Chennithala and seconded by Aryadan Mohammed. The election process was formally launched after Congress Working Committee (CWC) member Mohsina Kidwai and AICC general secretary Madhusoodan Mistry, who is in charge of Kerala, held a one-to-one meeting in their capacity as observers with the 38 elected MLAs.

UDF (United Democratic Front) led by Oommen Chandy secured a slender margin in assembly election which held on 13 April 2011. UDF candidates won in 72 seats against 68 seats of LDF (Left Democratic Front). He took the oath on 18 May 2011 with six other ministers of his cabinet. Later thirteen other ministers were also inducted into his cabinet.

He gave up Vigilance Portfolio in early August 2011, after a fresh probe was ordered on the Palm Oil case (which happened when he was Finance Minister in 1992).

Though UDF came to power with a wafer thin majority, his 100 days programmes announced propelled the state fast forward. 107 programmes were announced as part of the 100 days programme of the Government. Out of this the government could accomplished a whopping figure of 101 leaving just 6 of the programmes unfinished. This really gave a boost to the governance. The policies adopted by Oommen Chandy supposedly helped to bridge the gap between people and officials, although further evidence is needed.

UNDP appreciated Oommen Chandy, for the range of innovative practices in democratic governance, initiated by the Government of Kerala in strengthening people's access and participation in human development and governance, although later cases of corruption questioned the reality of such policies. They were impressed by the innovative approach to ensure transparency and accountability in Governance, particularly the to web-stream the entire functioning of CM’s office, which was turned off during supposed illegal activities. The Mass Contact Programme, in many ways, is the first of its kind to ensure the right to direct access to leaders and senior civil servants of the government.[3]

Oommen Chandy resigned as Chief Minister on 20 May 2016 following the defeat of UDF in 2016 Assembly Elections.

Oommen Chandy's office received the 2013 United Nations Public Service Award[4][5][6] from the Asia-Pacific region, for the category "Preventing and Combating Corruption in the Public Service." The award was presented on 27 June 2013, in Manama, Bahrain, by the UN Under-Secretary-General for Economic and Social Affairs, Wu Hongbo. The award was based on the theme "Transformative e-Government and Innovation: Creating a Better Future for All."