One woman has given her credit cards to her granny up the road to prevent her spending, while another has stopped paying into her pension but hasn't told her husband.

These are just two of the stories heard by Fran Walton, director at The Futures Company, the market research specialist which is part of the WPP group, as his team travelled the UK gathering data for its latest report into consumer confidence.

"We have interviewed people in Sheffield, Glasgow, Egham in Surrey and Staines to get a snapshot of the real picture. People revealed their levels of debt and their worries for the future. Sometimes there is a real sense of shame or resentment," says Walton.

But while the anecdotes are sad, they are not unusual. They show the often grim reality of consumers' daily lives and attitudes to shopping and spending around the country.

There is also a new savviness, whether from the 24-year-old unemployed former events worker in Glasgow who is so bored she does her sister's homework, or the retired widow thinking of sacking the gardener and cutting back her cleaner's hours, the message is the same – consumers want better deals.

The 1,014 people surveyed and the 12 in-depth interviews by The Futures Company make up the insightful data for its new report, "The Consumer Outlook 2011", published last Thursday, which offers a view of the consumer mindset for this year and makes predictions for next.

But is the picture really that bad? Last week's update from the British Retail Consortium appeared to offer some hope for retailers – revealing April's like-for-like retail sales had jumped 5.2 per cent thanks to warm weather and back-to-back long weekends, while Sainsbury's announced it grew its underlying pre-tax profits by 9 per cent to £665m.

But the retail outlook is patchy, with storms brewing for some: HMV is finalising plans to sell off its Canadian business in a bid to survive and is considering a company voluntary arrangement (CVA) among other routes in the UK, while electrical retailer Comet says that its like-for-like sales decline has reached 15 per cent since early January.

The higher commodity and fuel prices, the rise in the long-term jobless, the hike in taxes and a drop in government spending, along with the threat of an increased interest rate are all in the minds – and hitting the pockets – of the UK's shoppers.

On Friday, the Institute for Fiscal Studies revealed research that shows households may be facing their biggest drop in income for 30 years. Data shows that earnings undershot inflation by 3.8 per cent for the first 11 months of 2010-11.

But for the retailers that are suffering, the insight from the chats on the sofas in Sheffield, Glasgow, Egham and Staines can help them plan for the next 12 months.

Consumers are becoming increasingly cautious, with many people's real household income contracting for the first time since the 1980s.

The Futures Company has been keeping tabs on the consumer outlook nationally in a series of studies since July 2008. This latest has found that shoppers have begun to shift their spending patterns, becoming more concerned with finding the best value products and the best deals.

The report splits UK consumers into three categories, independent of social background or age, classifying people as either "plain sailing" – those who do not have worries about money or debt; "choppy waters" – those who have some problems but are managing overall; and "all hands on deck" – those who have serious debts and very little income.

Even those in the "plain sailing" category are cutting back, says Walton: "The plain sailing group are being smart and saving money because it is a sensible thing to do, even though they have money. They are not splashing out on luxury items as they don't want the stigma of spending when others are cutting back.

"Consumers are putting scrutiny on every purchase they make. Price inflation and wage deflation is affecting a broad spectrum of people across the population."

The research revealed further insights into shoppers' mentalities. Even those with debts are treating themselves on occasion. More than 60 per cent of those surveyed said they like to treat themselves even if they have to tighten their budget elsewhere to compensate.

So how can retailers do well in such a constrained environment? Walton and his team have a number of recommendations for them. He explains: "Retailers can acknowledge the tough times are not a blip, provide long-lasting value and give the shoppers a reason to hope. Promotions, vouchers, Groupon and guarantees on products all help people decide to spend."

The Sainsbury's launch of its menu on how to feed a family for a week for £50 is an attempt to ensure customers still spend with them, even when cutting back. While Marks & Spencer chief executive Marc Bolland noted the high uptake of its promotions in the past three months, including its Dine In for Two for £10 deal.

Kate Calvert, retail analyst at Seymour Pierce, has been following how retailers have coped since the recession. She says: "Housing related, discretionary and high ticket price categories have suffered the most, such as furniture, carpets and electricals. The value end of the market has done well. We have seen the growth of the pound-type store on the high street, and the discount food retailers have gained market share. Sports Direct has thrived at the value end of the sporting goods market. Value clothing retailers initially did well but then consumers switched to spending a little more on quality that would last, so companies such as M&S and Next have come into their own.

"Retailers who have done better are those which have embraced multi-channel, have a clear value proposition and a defined niche. Those who trade in the categories the food retailers are moving into, such as HMV, Game and Mothercare, have suffered."

Online shopping is also becoming more important. Calvert explains that there has been a push towards multi-channel so shoppers can choose when and how they buy and can compare prices. "Reserve and collect has been a great innovation." Most retailers have Facebook pages to raise their profile, and engaging with customers via mobile is likely to be the future for many retailers.

Karima Serageldin, a director at Ariadne Capital, a venture and advisory firm which invests in a number of digital businesses in the sector, explains: "There are a number of new apps and networked solutions being developed that will be able to track shoppers' preferences and offer deals or information based on previous purchases. For instance, Geocast delivers businesses a means to increase footfall while giving consumers an easy way to save money and 'shop local' via vouchers pushed to your phone at the point of consumption. For example, while you are outside Pizza Express you will receive a 2-for-1 lunch offer on your mobile phone."

Back on the sofas in Sheffield, Glasgow, Egham and Staines, the outlook remains pretty bleak. Nearly half the people surveyed said they feel worse about their financial situation over the coming year than last year. It is likely to get worse still for many before it gets better, so retailers must work harder for every last pound.