In 2013, I was visiting my elderly mother at her care home. “Mum, I’m writing a book about Jim,” I said – I could never refer to him as Dad. Her eyes widened as she put down her coffee cup. “Well, that’s interesting dear, but how will you fill a whole book? After all, you never even knew him.”

“I’m basing it on his diaries,” I replied, uncertain of her reaction.

She frowned. “What diaries? I don’t remember any diaries.”

“The ones he wrote in the war,” I said and hesitated. “I think you may have given them to me when you married Dick.”

I watched her face for signs of recollection or discomfort, but she merely shook her head and, avoiding my eye, responded, “I suppose I must have forgotten.” Then she added in a less than convincing tone, “Perhaps you’d show them to me sometime.”

Janet as a baby in the mid-1940s.

In 1970, my mother embarked on a second marriage after 27 years of widowhood. She deserved her new-found happiness. Nevertheless, I felt secure ground shifting beneath my feet. For 22 years there had been just the two of us, until I left her to marry my own love without considering the earthquake it must have caused in her life. In the years that followed the second world war she struggled, on a small income, to give me a happy fulfilling childhood. She had done a great job – guiding with a light touch, allowing me to live my own life, never making demands.

In fact, she had not given me the diaries. I had discovered two slender exercise books in faux leather covers among a pile of unwanted objects as I helped to clear her flat before her wedding. They looked interesting so I stuffed them into my bag. I still recall the feeling of joyful discovery when, after I had bathed and put my toddlers to bed, I flicked through the pages and realised what I had found. These were my father’s own thoughts, feelings and beliefs, written in his own hand. Did Mum know what they were when she abandoned them with other redundant items? After all those years of loneliness, could I blame her if she decided to wipe the slate clean of her first brief marriage in preparation for a new life? No, I could not. But I didn’t tell her what I had found. My life was busy so I put the books away to revisit at some later date.

Thirty six years later, I was clearing another house – the home I shared with my own family. The year 2006 was the worst of my life. We were still grieving the loss of our elder son to a brain tumour a few years previously when I cared for my husband as motor neurone disease inexorably deprived him of the ability to move, speak, eat, drink and, finally, to breathe. My life was very dark and lonely now. I was the sole occupant of our large 15th century farmhouse and about to downsize. I remembered that I had also lost a third man – the father who died on the day I was born. I recalled those long lost diaries. Could my dad help to fill the painful void in my life?

The picture of a young man in his Royal Air Force uniform had always stood in the centre of the mantelpiece of my childhood home. His father, my beloved granddad, told me he had been killed by the “bloody Germans”. His body was never found and for years after the end of the war there were reports of returning prisoners of war. I felt sure my dad would be one of them.

I longed to know more about him, but never asked. My mother and grandparents were everything to me. They were bulwarks against all that was out there in the wide world of ration books and bomb sites, and the ghosts who inhabited them. I was afraid of breaching this security fence; afraid my questions would pierce their emotional armour to reveal their frail humanity; afraid they would cry.

In 2006 I was triumphant when I found the books again. I poured a glass of wine and settled down in an armchair to read them. The entries began as war was declared on 3 September 1939. The words of a passionate, socialist, working-class young man of 18 leapt from the pages. Only weeks before he had spent a holiday at the home of his German pen-friend. Everyone knew war was looming but he had promised never to harm the family or their beautiful country. Back at home, he wrote that nobody would persuade him to fight. Nothing could be gained by killing.

My attention was gripped as I read his detailed account of the first month of the “phoney war”, his beliefs and his assessment of the causes of conflict. He describes attending meetings of the Peace Pledge Union and his family’s shock when he resigned from his job. The factory where he worked as an invoice clerk was manufacturing arms components and, “being an absolute pacifist I could not help with the armament programme”.

I always had an idea that, like me, Jim was a pacifist when war broke out, but it gave me a little frisson of excitement to see it proudly stated in his own hand.

After the entry of 20 September 1939, describing Poland’s fall, I turned the page to find nothing. The rest of the notebook was blank.

But I had the second volume and the first entry read: “Aberystwyth, Thursday 16 April, 1941.”

“It is many years since I kept a diary. When I was at school I made various attempts in the little pocket diaries that are the inevitable Christmas presents …”

So, he had forgotten those impassioned pages of writing 18 months earlier. He had begun a new diary for a new life. “… now events are moving so rapidly, so many things may happen to me in the next few years that I feel I must keep a record of events and my reactions …”

In the intervening months he had not only married my mother, Pat, his childhood sweetheart, but had volunteered for RAF Bomber Command. What changed his mind? Why did he reject his deeply held pacifist beliefs?

I needed to solve the mystery. I needed to follow him through the diary right up to the final entry in May 1943. I needed to find out what happened on the day of my birth to prevent me knowing my father. When I asked Mum why he renounced pacifism she replied, “I’m not sure, dear. We didn’t talk about things like that.”

Why? She is an intelligent woman. I was frustrated and had to remind myself how different relationships were then.

My uncle, Jim’s younger brother, was more helpful. He told me that the family lived near the River Thames on the edge of “Bomb Alley” and witnessed the devastation of the Blitz on London. Jim had seen his parents terrified, neighbours buried under the rubble of their homes and began to feel uneasy about his position. However, it was when Hitler invaded Russia – utopia for ardent socialists in the 1930s – that Jim reluctantly decided that he must fight to stop him. At heart he remained anti-war, repeatedly questioning his decision. As a volunteer he could have got out at any time, but he writes, “In my heart of hearts I know that fascism must be crushed.”

Using his diary as a guide, I decided to walk in Jim’s footsteps. I imagined his feelings at Lord’s cricket ground as he gathered with other raw recruits, in Aberystwyth, where I stood in his bedroom, at Ansty where he flew for the first time and in Dumfries, where he learned the dark art of air-bombing. Finally, I found the Yorkshire airfield from which he took off. I lay on the bomb-aimer’s couch in a restored Halifax II and looked down through the transparent nose. How had he felt when he pressed the button to release his bombs? The Ruhr valley, burning below him was where his pen-friend lived.

When my father’s plane turned for home on his final mission, my mother was in labour and knew nothing of the night-fighter attack over the North Sea that claimed his life.

I was born a few hours later. She never saw him again.

Searching for information about his last flight, I typed his plane’s number into the Aircrew Remembrance Society website and, unbelievably, up came a picture of the German crew who had shot it down. Pleasant looking young men, they themselves were lost a few months later. I found myself weeping for the loss of so many who were only doing a job.

The two men on the right were in the Luftwaffe aircrew that shot Jim’s plane down in 1943 – they were killed themselves a few months later.

I have been asked if discovering my father was an emotional experience. My answer is, yes! But the emotion is not grief or sadness – how can I grieve for a man I never knew? Rather, it has been excitement at finding who he was; who I am.

I have shared his agonies of conscience, understood the depth of love he had for Mum and for me, his unborn child. I have wondered how the last seven decades might have changed him if he had seen how the world would change. I know how strongly I resemble the man in the photo on the mantelpiece. Now I have come to wonder at other genetic inheritances that live on in me, his daughter, and in his grandchildren and great grandchildren – our passions, our politics, our interests, our ambitions.

When he is reflecting on the risks of combat, Jim writes: “I must come through this alive and well. And if I don’t? Well, maybe those dearest to me will like to keep this diary and perhaps find me still living in these pages …”

I found my dad and he came alive for me. He gave me a reason to look to the future and know that all my three lost men can live on in the memories of others.

The Man on the Mantlepiece by Janet Denny is published by Silverwood Books, £9.99. To order a copy, go to bookshop.theguardian.com or call 0330 333 6846