There’s a Homeric hymn that sings of a mortal man beloved of one of the gods: Eos was goddess of the dawn, and so enamoured was she of Tithonus that she couldn’t bear that one day he would die. She begged Zeus that her lover be granted immortality. “Too simple was queenly Eos,” the poem goes on, “she thought not in her heart to ask youth for him, and to strip him of the slough of deadly age.”

The two lovers lived in great happiness for many years until Tithonus began to show signs of age. At first Eos banished him from her bed, though she still loved him, and “cherished him in her house and nourished him with food and ambrosia”. Tortured by his immortality, Tithonus’s body continued to fail: he became so weak that he could not even lift his withered limbs. Eos was unable to cure him of his miseries, but because of Zeus’s intervention she was equally unable to let him die. Finally, she “laid him in a room and put to the shining doors. There he babbles endlessly, and no more has strength such as once he had in his supple limbs.”

The Greeks chose their myths with care; this one illustrates two dilemmas about death and ageing that in the west we are only now beginning to grapple with. The first is how we plan to care for loved ones who, though their bodies and minds are weakening, are granted great longevity. The second dilemma is how we manage the body’s decline when a “cure” is out of reach. Through the two halves of his book, Atul Gawande – Boston surgeon, staff writer for the New Yorker, this year’s choice for the BBC Reith lectures – has written an impassioned, broad-ranging and deeply personal exploration of both.

The first dilemma concerns the tremendous success of modern healthcare: the infections that once killed most of us have been subdued, and the diseases that even a few decades ago killed us in our 60s and 70s – cancer, heart disease, stroke and emphysema – are increasingly treatable. It is now common to live into your 80s and even 90s, ages that in human history have been the exception rather than the rule. We know how to prolong the life of the body, but we still have little idea how to slow the ageing process or prolong the life of the mind. Even for those who avoid dementia, the weakness and frailty of extreme old age bring a reliance on others that often requires admission to a nursing home.

It’s a paradox that innovation-driven, individual-orientated societies have evolved medical technologies and therapeutics so advanced that we can keep people alive for longer than ever before, but those same societies are not set up to care for those who become dependent through old age. Sociologists tell us that prosperous non-western societies develop along similar lines: when people gain the financial means to live independently from their family, they choose to do so. Most elderly people, when they themselves become dependent, concur with this belief: the wish to avoid “being a burden” on the next generation is a powerful and transcultural one. Gawande explodes the myth that traditional societies are always more respectful towards the old.

Before the postwar settlements and introduction of welfare states, those who reached an age advanced enough to require assistance were looked after by their family; if there was no family, they went to an almshouse. These were unregulated and often squalid (as the ones Gawande visits in India still are), accelerating the decline of those admitted. The social and medical advances in the 20th century led, in the western world, to the rise of hospitals, which gradually began to admit those with the irremediable problems associated with ageing. Nursing homes were devised specifically to manage this population; as institutions, they have their origins in a medical model of care. “Nursing home” has two words to it, and two aspects, but many prioritise the provision of “nursing” over the creation of a “home”. In his 1961 study Asylums, Erving Goffman noted the shocking similarities between nursing homes and prisons. In common with orphanages, mental hospitals and military barracks, they were “total institutions”, preoccupied with minimising risk and facilitating staff routines at the expense of granting adults autonomy and self-respect. Even now, there are many nursing homes in which clients are not allowed to walk alone in case they fall, eat certain foods they want in case they choke, use knives in case they cut themselves. In many of the institutional homes I visit as a GP, residents are not allowed to keep pets in case those pets inconvenience staff, or bite their owners. As Gawande notes in one of the many poignant passages in this book, we permit children to take more risks than these adults, who are presumably old enough to make their own choices – children at least are allowed to play on climbing frames.

An obsession with risk is stultifying the lives of the most vulnerable in our society, in the years when their choices should be most cherished and respected – even if those choices shorten their lives. We have exalted longevity over what makes life worth living. More chillingly, Gawande shows how infantilisation of the old is promoted by profiteering companies. Building more compassionate institutions is not only straightforward, it costs nothing more, and the benefits in terms of improved quality of life are immense.

If the first half of his book concerns nursing homes and how we can age with self-respect, the second half concerns palliative care and how we can die with grace. The stunning victories of medical science over the last century have, according to some critics, left too many doctors arrogant and unwilling to concede defeat (the militaristic cliches are essential to this vision of medicine and the body). We’re waking up to that mistake now: hospice and palliative care are at last receiving the attention and the funding they deserve; helping our patients through a good death is increasingly acknowledged to be as important as helping them flourish while alive. Much of the second half of the book concerns the health system of the US where Gawande practises, and where the hospice movement and the provision of community palliative care are relatively undeveloped compared to those in the UK.

In a moving series of reflections, he reveals just how terrible doctors are at telling patients when treatments are unlikely to work, and how hopeless they are at estimating how much time their terminally ill patients have left. It’s one of the questions I’m most commonly asked after breaking the news of a terminal diagnosis, and one that is almost always impossible to answer, but I was shocked when Gawande revealed that doctors don’t just routinely get it wrong – when asked to predict how long a terminally ill patient has to live, they overestimate on average by a factor of five. The intrusion of commercialised medicine, and the elevation of the interests of insurance companies over those of patients, can complicate these issues considerably, but Gawande remains clear-sighted through the muddle of anxieties, conflicting emotions and vested interests.

Towards the end of the book, he tells the story of his own father’s decline and death from a tumour of the spine. His experience as a surgeon melts away and he finds himself navigating infirmity and dependency as a son, rather than as a clinician. It’s the worried son, not the Boston surgeon, who reflects on the qualities he values in the doctors treating his father: not bullish arrogance, but acknowledgement of uncertainties and a willingness to accept risks. He finds doctors communicate most effectively when they jettison the position of detached, clinical observers and talk in terms of how they feel: “I am worried about your tumour because … ” Often the bravest and most humane decision, he realises, is to do nothing at all.

When time becomes short, Gawande has the presence of mind to ask his father: “How much are you willing to go through just to have a chance of living longer?” The answer helps guide his father to a relatively peaceful death in the arms of his family, as opposed to a technologised end on an intensive care unit. The message resounding through Being Mortal is that our lives have narrative – we all want to be the authors of our own stories, and in stories endings matter. Doctors and other clinicians have to get better at helping people with their endings, otherwise more and more of us will end our lives babbling behind shining doors.

Gavin Francis’s Adventures in Human Being, his book about medicine and the body, is out next May. To order Being Mortal for £13.59 (RRP £15.99) go to bookshop.theguardian.com or call 0330 333 6846.