A little girl rides on her father’s shoulders at a book fair in Kolkata.
Photograph: Pacific Press/Getty Images

Somini Sengupta was eight when her family left Kolkata to make a new life in California. She spent her childhood travelling between the two countries, before returning to India in 2005 to work as the New Delhi bureau chief for the New York Times. She currently lives in New York. Her book The End of Karma: Hope and Fury Among India’s Young, explores the “new India” through the stories of seven individuals, from the ambitious son of a rickshaw driver to a young woman who becomes a Maoist rebel.

When you returned to India in 2005 did it feel like coming home?My family is from Kolkata and so during my childhood every three or four years we would go there for summer vacation, but I didn’t really know a great deal about the rest of India. When I first landed in Delhi, a friend asked me, “Does it feel like home?” and I was quite adamant in the beginning that it didn’t. But after four or five years, when I became a mother, that’s when I really discovered how much at home I did feel. My daughter was born in India and she was completely at home in our neighbourhood and that’s when I think my emotional stake in India and also in this book deepened. I kept wondering if she would some day ask the question I sometimes ask, which is “Who might I have become had I stayed in India?”

How did the idea for the book come about?India has the largest concentration of young people in the world at any time in recorded history – 420 million Indians are between the ages of 15 and 34, and every month for the next several years, 1 million Indians are turning 18. That has a remarkable impact not only on the Indian economy, society and politics, but it’s going to shape the world for the rest of us. It will have a bearing on everything from migration to efforts to combat climate change. I wanted to take a look at India through the eyes of the young. While India has always been a young country, what has shifted in the last couple of generations is that Indian women are having fewer babies and so the percentage of young children in the population is steadily falling and also the percentage of elderly people is fairly small compared to many other countries, and then there’s this big bulge of working-age people. Of course it’s an incredible boon for any country to have this so-called “demographic dividend”, but it can also create very high expectations, huge demands and, also, frustrations when expectations are not met.

What do you mean by “the end of karma”?This generation of young people are all, in their own ways, trying to overcome their past. They’re trying to write a new destiny for themselves. The dominant theme [of the book] is ambition and all the ways it’s thwarted by state policies – such as the failure of schools, or high levels of childhood malnutrition – or thwarted by society: the notion that women are lesser and should be restricted, or notions about caste and untouchability that still persist.

India’s caste system still seems remarkably entrenched. You quote a statistic that fewer than 5% of Indians marry outside their caste, for example. Do you see this changing any time soon?When the Indian constitution was adopted it promised equality in the eyes of the law, no matter what your caste, gender or religion, and that to me is a pretty remarkable and audacious promise for the founders of India to make in such a highly stratified society. Caste does not have the original currency that it once did, which is to say that it doesn’t necessarily define what you do for a living, but it’s still very important in terms of shaping people’s identity, in marriage and in voting patterns. It also has been a source of hand-wringing for the nation in recent months after the suicide of a Dalit student in one of the universities. It lead to a lot of soul-searching about how the Dalit, who were formerly referred to as “untouchables”, remain discriminated against.

One of your case studies looks at the so-called “honour” killing of a couple who married across caste lines. The most shocking element of the story is that the crime was committed by their peers – young, urban Indians.I think some of the old social hierarchies don’t just go away with modernisation. These beliefs can thrive even in a modern city like Delhi. As a parallel, the longstanding preference for sons also continues and actually has gotten worse as India has modernised, because now people can afford ultrasound tests and they can more easily abort female foetuses. So it was worth telling that story precisely because these ideas of who should marry whom, how much freedom a woman should have, these were social mores that were still very much being enforced by young people.

You write about the “degradation of the female” in Indian society starting in the womb.It was something that I really thought about because my daughter is a daughter of India and I have been struck by how this audacious promise was made at independence to treat women as equals, but how that promise has not really been met yet. But I think that the outcry you heard a couple of years ago about the gang rape in Delhi was a real watershed moment. What I really sensed from those protests in 2012 was that women were not going to take it any more. There was a huge public outcry, to draw attention to this issue and to force a change in the laws and to encourage women to speak out about sexual harassment and violence. Shortly after his election, the prime minister Narendra Modi made a remarkable speech about parents not just having to keep an eye on their daughters, but also goading them to keep an eye on their sons. It signalled that politicians are having to take this seriously, but having said that, there’s not a whole lot that his government has done to change laws and policies around violence towards women either.

What is the biggest misconception that the rest of the world has about India?There are so many! One of them is the myth that India is this bottomless well of clever, hardworking maths nerds who are about to steal our jobs. The reality is that the vast majority of young Indians are in no position to compete in the global economy. Many of them have been so cheated.

Your book highlights how the education system fails many children.That is one of the huge fault lines of India at this moment. Surveys have shown that half of fifth-graders (those who have nominally stayed in school for five years) can’t read a basic text book or do basic subtraction. So while it’s very promising that so many more Indians are going to school than ever before – primary school enrolment is almost 100% – the quality of educational outcomes remains very poor.

Children take an exam on the first day at a technical high school at Ghatkopar, Mumbai. Photograph: Getty Images

The statistic you quote that one in four Indians lives on less than $1.25 (87p) a day is all the more shocking when set against the country’s recent spell of economic growth. What’s going on?The percentage of Indians living below the poverty line has gone down sharply and indicators of social wellbeing like life expectancy and infant mortality have also improved. But there are some really worrying indicators: such as child malnutrition. In such a huge economy that’s been growing at 6 or 7% a year for so many years, it is really puzzling why 30% of children under five remain clinically stunted. The inability to address child hunger is in my view one of the most shocking facts of India’s economic growth.

At one point in the book you describe India as an “unlikely nation” – due to its mix of races, cultures, languages and religions. Is it a miracle that it’s held together for so long as a democracy?When India became independent and chose a federal democratic path, there were many naysayers who said it can’t hold, it will break up, especially without a dictator at the helm. With the exception of a two-year period of emergency when democracy was suspended, in fact it has survived and that in some ways is a really good object lesson to all those who predict the splintering of other countries in the world today, namely in the Middle East. India has shown a different way to be a democracy. However, it has its moments of identity crisis and I think it is living through one of those moments right now when it has to contend with what kind of a secular democracy it’s going to be, how religious minorities will be protected, what room there is for political dissent, how much free expression will be tolerated.

Are you optimistic about the future?India’s youth bulge is a historic opportunity. India can thrive or implode – or indeed, both. It would be folly to try to read the tea leaves for a country like India. But I couldn’t have stayed with this project were I not hopeful about the power of the individual, dynamic and ambitious young people I met to show the world a way forward.