I grew up in a mining village in the West Riding of Yorkshire. Dad was a colliery electrician but also ran a smallholding growing vegetables and keeping poultry.

When his foot was crushed in a mining accident, he went into poultry farming full-time before starting a small grocery shop with Mum. He died when I was 14, but I think I inherited my work ethic from him. Even at school I wanted to be a businessman and organised dances around Huddersfield, booking popular live bands.

I once got an offer from a guy in Sheffield wanting to come in on dance events with me. His name was Peter Stringfellow. Life could have been a lot more interesting.

I didn’t do very well at school and failed to get into Marks & Spencer, Littlewoods and John Lewis, but Woolworths would take anybody. I started at the bottom, sweeping the stockroom floor, hoping to become a director. Two years later I was appointed trainee manager, but it was a nominal title as there was no formal training programme. I was a forgotten man.

Have you always been an entrepreneur?

One of my jobs in Woolworths was weighing seed potatoes. If there were too many, I took some home to our smallholding, planted them and then sold them on to the canteen at Woolies. I had an inherent desire to make money from the start.

How did you launch Iceland?

I opened the first shop in Oswestry in Shropshire with another disillusioned Woolworths worker in 1970. We didn’t have any savings, so we decided to specialise in fruit and veg because it was the only business we could think of where you could buy stock daily and pay at the end of the week. Then I saw a department store in Leeds selling loose frozen foods, so we copied them. My wife, Ranny, came up with the name Iceland. We nearly called ourselves Penguin.

How did you expand the business without savings?

We put down a deposit of £30 each to cover the first month’s rent, and bought two open-top freezers, scales and a cash register, all on hire purchase. Then we found a local food supplier who gave us credit for a week, which was then stretched to a month. Three months later Woolworths discovered what we were doing and fired us, but this simply gave us the incentive to press on and open more shops.

What has been your biggest mistake?

In 1996, Iceland had its first profit setback, dropping from £73m to £65m net profit . We’d grown complacent. If we had been a private company it wouldn’t have mattered, but by then we were a public company and that little dip was viewed as the end of the world by our shareholders. Our market value dropped from £800m to £160m. Our track record of 25 years of increasing profitability meant nothing.

You were forced to resign in 2001 after a merger with the Booker Group, Britain’s largest cash and carry operator, amid accusations of improper share dealing. How did that feel?

It was horrendous. From being the guy who had built a business from nothing, I became the crook who had sold shares ahead of a profit warning. I was planning to retire. Both the Financial Services Authority and the Department of Trade & Industry launched inquiries and eventually, in 2004, I received a letter by second-class post informing me that I’d been cleared.

What has been your largest investment?

Borrowing £800m to buy back Iceland for the second time in 2012. I again became chairman and chief executive after leading a successful £1.5bn management buyout with the support of three new equity investors.

Almost 42 years after I started the company, it was back in private hands. When we bought it in 2005 we paid £160m.

When we bought it again in 2012 the value had increased to £1.5bn.

Has your BBC television outing in Iceland Foods: Life in the Freezer Cabinet been good or bad for business?

There were many cringeworthy moments, but on balance it worked. The feedback has been positive and sales of some of the products mentioned, like the Bubble Bobble King Prawns, have gone off the scale. But in my heart of hearts I do know that elements of that series haven’t done us any good at all.

When I saw the first edit I was horrified. If I’d edited the programme myself I wouldn’t have emphasised the downmarket side of the business quite as much. We certainly won’t be attracting any Waitrose shoppers.

You’ve been critical of the £2.5m tax bill Iceland received after taking 800 managers on a dream trip to Disneyland. Have there been any developments?

We’ve since settled with the Inland Revenue and we’re very happy with the outcome, though I’m not allowed to talk about it.

The taxman made concessions because we argued, successfully, about the investment value of the trip as opposed to the entertainment value. We do this to boost morale, it’s not a holiday. We certainly haven’t been put off. We’re taking our managers to Dubai next year.

What annual salary do you take from the business?

Enough.

Your personal wealth has been put at £215m. Is this accurate?

That’s the rich list figure, but it’s family wealth. My kids have got most of my money. It’s what you do, isn’t it? Diversify your wealth.

The media have made negative comparisons between your wealth and that of your customers. Does that bother you?

Why is it an issue? My personal wealth is nothing compared to the wealth we’ve created. My hunger for business has resulted in 25,000 jobs and £600m in tax contributions in the past six years alone.

Amazon, Google and Starbucks combined haven’t paid a fraction of the tax we’ve paid. Just think how many schools and hospitals you could build with that. My wealth isn’t something I should ever have to apologise for.

Does money make you happy?

I’d rather be miserable with it than without it.

How do you prefer to pay – cash, card or cheque?

It depends on how much the bill is. I don’t carry much cash. In the staff canteen I often have to ask other people to buy me lunch.

Do you bank online?

I don’t bank at all. My secretary sees to all that.

How much property do you own?

My main residence is Broxton Old Hall, a Grade II listed Elizabethan manor house in Cheshire, which I bought for £750,000 in 1985. It was in poor repair, so I knocked it down and rebuilt it. The renovations cost a lot more than I budgeted for.

I also have a holiday home in Majorca and a house in Chelsea, which I bought in 2002. We hardly use it, but it’s probably the best investment I’ve ever made because it has trebled in value.

What is the biggest drain on your finances?

Personal staff – gardeners, house staff and boat crew. I have an 82ft Oyster yacht, which I keep fully crewed, though I spend only two or three weeks a year on it. As we speak, it’s on the way to the Caribbean.

Do you plan to retire?

I’ve decided I never want to retire. Now that we’re a private company again it’s completely different and there are no outside pressures. I’m in this for life and completely focused on Iceland.

- 'Best Served Cold: The Rise, Fall and Rise Again of Malcolm Walker’ is published by Icon Books at £25