Narendra Modi, chief minister of the western Indian state of Gujarat, is no stranger to controversy. In 2002, anti-Muslim riots in Gujarat caused the deaths of over a thousand people. Modi and his administration were accused of standing idly by as the carnage was perpetrated — an allegation that he and other officials have steadfastly denied. In recent years, Modi has built a reputation as a can-do leader who has made Gujarat a business-friendly economic powerhouse. Now, Modi is even spoken of as the flag bearer of the opposition Bharatiya Janata Party (BJP) in national elections next year. If the BJP win, Modi could even become Prime Minister.

Yet his past continues to dog him. On Monday, the Wharton India Economic Forum (WIEF), an annual India-centric conference hosted by the Wharton School of the University of Pennsylvania, canceled the keynote address Modi was invited to give later this month, via videoconference. His speech was opposed by several Indian-American professors, who gathered more than 200 signatures to stop the event. “Our team felt that the potential polarizing reactions from subsegments of the alumni base, student body and our supporters might put Mr. Modi in a compromising position, which we would like to avoid at all costs, especially in the spirit of our conference’s purpose,” the WIEF organizing committee said in a statement.

It’s not the first time Modi, 62, has encountered friction overseas. In 2005 the U.S. State Department refused him a diplomatic visa to enter the country. The E.U., as a bloc, also blocked him from visiting its member nations. Germany and Britain have now lifted their ban, but the U.S. has yet to revise its policy. “We cannot give platforms to people who have violated so many human rights while being in power,” Ania Loomba, a Penn English professor and one of the petitioners, tells TIME. “It’s shocking that Indian students at Wharton feel that someone who is one of the most controversial figures in India should be invited to celebrate India’s ‘development’ and economic success.”

The turn of events didn’t surprise many in India, where Modi has many detractors. Last month he faced protests when he delivered a speech at New Delhi’s prestigious Shri Ram College of Commerce. His efforts to portray himself as secular and progressive, with a focus on “propeople good governance,” have been questioned by critics, who dismiss it as a public relations exercise. In Gujarat, however, Modi is a hero. The state boasts India’s lowest unemployment rate, and billions of dollars in investment are flowing in. Many Indian as well as foreign business leaders consistently praise him.

The rejection by Wharton might only be a small irritant in the grand scheme of Modi’s political ascension, but it highlights how radioactive he remains. “Owing to a certain kind of populist initiative, Modi could possibly become a national leader,” says Shiv Visvanathan, a social scientist with New Delhi’s Jindal School of Government and Public Policy. “But he still has a long way to go, both within the BJP and the nation, to achieve that.”