Kahlon became politically active in the late 1980s, when he helped Rami Dotan campaign for mayor of Haifa. It was in the context of this election campaign that he met Uzi Landau, who appointed him his Bureau Chief when he became Minister of Public Security in 2001. Kahlon served in this position for a year. Kahlon was first elected to the Knesset in the 2003 elections, and was appointed Deputy Speaker of the Knesset. In the run up to the 2006 elections, he won third place on Likud's list in the party's primaries.[8] He retained his seat again in the 2009 elections after being placed sixth on the Likud list, and was appointed Minister of Communications on 31 March.

In the Knesset, he worked to pass a bill to reduce electricity charges for poor families and headed an inquiry into bank fees.[9][10] Kahlon was also credited with leading the "Cellular Revolution", a set of moves that allowed new competitors to enter the cellular communications market in Israel, including Golan Telecom. This drastically reduced cellular communications prices in the market.[11] On 19 January 2011, he was appointed Minister of Welfare & Social Services after the resignation of Isaac Herzog.

Kahlon announced he would be taking a break from politics,[12] and did not run in the 2013 Knesset elections.[13] In response to reports that he was going to form a new political party, Kahlon announced on 3 November 2013 that he would not.[14] It was subsequently reported that Kahlon would initiate a new party to run in the next Israeli legislative election; possible running-mates were reported to include Yoav Galant and Meir Dagan.[15] In April 2014, after a period of silence, Kahlon announced in an interview with the newspaper Yedioth Ahronoth his intention to return to politics "imminently", but that he had not decided on a "framework" for his return; in the same interview, he criticised the socio-economic and diplomatic policies of prime minister Benjamin Netanyahu, adding to speculation that he would attempt to run against Netanyahu in the future elections.[16] In 2014, he indeed founded a new political party, the Kulanu party, ahead of the expected March 2015 elections. He announced several new members of his Knesset list: former ambassador to the US Michael Oren, Israel Prize winner Eli Alaluf, Yifat Sasha-Biton, a former deputy mayor of Kiryat Shmona and Deputy Jerusalem Mayor Rachel Azaria.[17]