"You want to run for president?" asked the New York Times columnist Frank Bruni in his book Ambling into History. "Here's what you need to do.

"Have someone write you a lovely speech that stakes out popular positions in unwavering language – and less popular positions in fuzzier terms. Better yet, if it bows to God and country at every turn: that's called uplift. Make it rife with optimism, a trumpet blast not just about morning in America but about a perpetual dazzling dawn. Avoid talk of hard choices and daunting challenges; nobody wants those. Nod to people on all points of the political spectrum … Add a soupcon of alliteration. Sprinkle with a few personal observations or stories: it humanises you. Stir with enthusiasm."

So it was at the beginning of the year, as the Republicans competed to see who could paint the gloomiest picture of Barack Obama's America, that Obama reached back for the signature theme of his 2008 campaign: hope. Seeking to channel Ronald Reagan's re-election theme of 1984, when the nation was emerging from economic crisis, he used his state of the union speech in January to claim: "America is back". "Anyone who tells you that America is in decline, or that our influence has waned, doesn't know what they're talking about," he said.

He test-drove the phrase in a range of settings. "I placed my bet on the American worker," he told a union conference in DC a month later. "The American auto industry is back." A month after that, at a fundraiser in Houston, he told donors: "The recovery is accelerating. America is coming back."

This was by no means an absurd claim. By February there had been three straight months of employment growth; the final quarter of 2011 showed a spike in consumer borrowing, signalling more consumption and more lending. In the spring, many felt they witnessed the green shoots of economic recovery. And electorally it seemed like a smart claim, too. American voters may want politicians to ponder their fragility, but they've never been particularly keen on them actually reflecting it. The country was emerging from two failed wars and the most severe recession since the Great Depression. Confidence in America's political and financial classes was shattered; assumptions about its military supremacy were dented. According to Gallup, the last time most Americans were satisfied with the direction the country was heading in was January 2004 – and that stint of optimism lasted less than a week. The Republicans were wedded to the notion that under Obama America was in decline. Rick Santorum claimed it was an election "to save the soul of America" – prompting the question: well, then, who had lost it? – while Mitt Romney insisted he'd return the nation to a day when "each of us could walk a little taller and stand a little straighter". Obama's message was: "Walk tall. We're on our way."

There was only one problem. People did not believe it. In February, Greenberg, Quinlan, Rosner Research focus-grouped four different ways of framing the nation's economic trajectory. Two concentrated on the enduring struggles of American middle-class families, and two claimed recovery was under way. The two that did best argued: "This is a make-or-break time for the middle class, and for all those trying to get into it." The one that tested worst, by a considerable margin, claimed: "America is back".

"America is not back," Stanley Greenberg, GQR's chief executive, told the New York Times. "We have long-term fundamental problems. If you look at our data and history, it takes a long time before job numbers translate into accepting at a personal level that things are better."

Indeed, if anything, Americans felt they were going backwards. Most believe young people will have a worse life than their parents, and a third think the country's best days are behind it. It's not difficult to see why. For 90% of Americans, wages have been effectively stagnant for the last 40 years, while median house prices have slumped 20% since 2006. Over the past decade, the cost of tuition has risen 32%, and the average healthcare premiums have rocketed 113%. A report earlier this year revealed that between 2007 and 2010, the median American family lost a generation of wealth. With figures like that, insisting that America is back sounds more like happy talk than fighting talk.

Barack Obama: standing not on his promise, but on his record. Photograph: Getty Isaac Brekken/Getty Images

Herein lies the central dilemma for Obama's re-election. In 2008 he ran, with considerable rhetorical force, on a promise of hope and change in the midst of an economic crisis – and on his ability to bring consensus to a divided political class. But, for many, things have changed for the worse, and the country is even more polarised than when he started.

So the substantial benefits of his presidency are not fully apparent – particularly to those most likely to have voted for him. Meanwhile, the symbolic significance of his candidacy is largely spent. You can only be elected the first black president once. His presence remains a source of great pride to many, particularly African Americans and the young. People will still travel halfway across the country on their own dime to hear him speak, and hawkers still sell T-shirts at his events. When he went to Fort Myers in Florida to speak in July, people started lining up for tickets the night before.

But this time he's not standing on his promise, but on his record. Shortly after his inauguration, he told NBC: "Look, I'm at the start of my administration. One nice thing about the situation I find myself in is that I will be held accountable. You know, I've got four years. And a year from now I think people are going to see that we're starting to make some progress.

"But there's still going to be some pain out there. If I don't have this done in three years, there's going to be a one-term proposition."

The pain is still out there, and that is precisely the proposition the Republicans are now making.

***

There are several ways Obama can counter this. When he first ran, few understood the depth of the economic crisis, and few could have predicted the implosion of the eurozone and the subsequent drag on the world economy. Roughly two-thirds of the country still blame Bush for the state of the economy, while only half hold Obama responsible. Republicans have been both obdurate and obtuse in Congress, where approval ratings have rarely scraped 20%. The trouble is, to a sceptical ear, these sound more like justifications for why he has failed to deliver than explanations as to how he might succeed if given more time.

Bill Clinton conceded as much during his convention speech in Charlotte. "Here's the challenge he faces," he said. "A lot of Americans are still angry and frustrated about this economy. If you look at the numbers, you know employment is growing, banks are beginning to lend again, and in a lot of places, housing prices have even began to pick up.

"But too many people do not feel it yet. I had this same thing happen in 1994 and early '95. But the difference this time is purely in the circumstances. President Obama started with a much weaker economy than I did. Listen to me now: no president could have fully repaired all the damage that he found in just four years. If you will renew the president's contract, you will feel it. You will feel it. Folks, whether the American people believe what I just said or not may be the whole election. I just want you to know that I believe it. With all my heart, I believe it."

The trouble for Obama is that not enough Americans do believe it. Stewardship of the economy is the one area where Romney has been generally outpolling him (although it has been slipping since the conventions). It is also by far the most important issue in the election. In the nine states Obama won from Republicans last time, his approval ratings are below 50; in six they are 45 or less. Unemployment is higher in six of them today than it was when he was inaugurated. In the midterms, Republicans took more than 20% of the House seats from Democrats in those states. His defeat is not just plausible; it's possible. Indeed, given the metrics of unemployment, approval ratings, real disposable personal income growth per capita (what the average person has left after tax and inflation) he should lose.

And if that weren't enough, Republicans will have far more money. In 2008, Obama outspent McCain by more than two to one. He has also outspent Romney so far, although that is largely because Romney had to wait until he was formally nominated before dipping into reserves of cash. But thanks to changes in the fundraising laws that allow unlimited donations from anonymous sources, Democrats fear being massively outspent by Republicans and their supporters this time. Obama donors at the convention were asked to open their wallets with the message: "If they outspend us by two or three times we're OK. But if it goes much beyond that we're in trouble." After a successful convention they came back thrilled by the speeches and daunted by the prospects.

"If the economy is doing great then any leader looks great," says one of Washington's premier electoral analysts, Charlie Cook. "And if the economy is doing lousy I think almost any leader looks bad." The day the Democratic convention opened, a poll showed that most Americans believe the country is worse off than it was when Obama was nominated four years ago, and that he does not deserve a second term. The day after it ended, the Treasury released a weak jobs report signalling a feeble recovery that's close to stalling.

The Situation Room in the White House. Obama has portrayed the killing of Osama bin Laden as a foreign policy success. Photograph: Pete Souza/AP

The problem is not that he doesn't have a record. He can point to significant achievements that would or should satisfy his base and arguably credit him with the most progressive term since Lyndon Johnson. He has appointed two women – including the first Hispanic – to the supreme court, implemented a version of healthcare reform, ended don't ask, don't tell, withdrawn combat troops from Iraq and presented a timetable for withdrawing forces from Afghanistan. He authorised the killing of Osama bin Laden, and the US's standing in the world has greatly improved since he took over among both allies and enemies alike.

The problem is that the record Obama has does not include the single most important achievement he could hope for – improving the economic lot of the broad swathe of middle America. And the various things it does include do not add up to a narrative. (It doesn't help that the main advantages to his principle achievement – healthcare – do not kick in for another two years). This was evident at the convention, where the same laundry list of Obama's achievements (the Lilly Ledbetter Act, which protects equal pay for equal work for women; the repeal of don't ask, don't tell; Bin Laden's killing; healthcare; his executive order on young immigrants) were aired each night less as a theme than an incantation.

It is a flaw best illustrated in Joe Biden's claim to donors in Fort Worth, Texas, that: "The best way to sum up the job the president has done if you need a real shorthand: Osama bin Laden is dead and General Motors is alive." Among other problems, such a summary links a single military operation and a particular economic achievement, both of which mark progress that is both episodic and partial. A counter-summary could just as easily read: "Al-Qaida is alive and the economy is dying." Moreover, it falls well short of "We're back," (which is what people want to hear), let alone "hope" and "change" (which is what they heard last time).

The greatest case for Obama's presidency so far can be summed up thus: things were terrible when I came to power, are much better than they would have been were I not in power, and will deteriorate if I am removed from power. Even if one accepts these claims as true, and understands them as important, they're a long way from the uplifting message of four years ago. Not so much "Yes we can," as "Could be worse".

The problem with Obama trying to echo Reagan's lines of 1984 is that Republicans can and do counter with Reagan's line from 1980, when, concluding a debate with Jimmy Carter, he asked: "Are you better off than you were four years ago? Is it easier for you to go and buy things in the store than it was four years ago? Is there more or less unemployment in the country than there was four years ago?"

At the Republican convention in Tampa, this was evoked as the game-changing line gifted to Romney by history. But it only works if Reagan's quote is hacked in half. For the Gipper immediately went on to ask: "Is America as respected throughout the world as it was? Do you feel that our security is as safe, that we're as strong as we were four years ago?" Even with American embassies under siege, most here would give him credit for that.

It is rare for a president to recover from this level of protracted economic distress, particularly when they brought such high hopes with them. But the last year handed Obama two crucial, mutually reinforcing tools with which he could start to build an electoral revival. The first was Occupy Wall Street, which sprouted offshoots in every state in the country, burning brightly before fading into smaller more grassroots campaigns. The occupations had no specific demands, and had no organic connection to the Democratic party. But by concentrating their ire on the inequities of the financial system and the greed of financial elites – two things Obama had failed to do anything about – they shifted the target of national frustration from government to inequality.

Polls showed a significant portion of the country agreed with its aims, with 77% believing there is too much power in the hands of a few rich people and corporations. Even as rightwing pundits and politicians mocked the protests ("Get a job right after you take a bath," said Republican contender Newt Gingrich), conservative analysts noted that they had struck a chord. When rightwing pollster Frank Luntz addressed the Republican Governors Association in December, he told them: "The public still prefers capitalism to socialism, but they think capitalism is immoral. And if we're seen as defenders of Wall Street, we've got a problem."

Obama had clearly got the message. In the same state of the union speech in which he proclaimed the US to be back, he appealed directly to the 98% of the country that earns less than $250,000. "Let's never forget," he said: "Millions of Americans who work hard and play by the rules every day deserve a government and a financial system that does the same. It's time to apply the same rules from top to bottom: no bailouts, no handouts, and no cop-outs."

And while there is no narrative in which to couch his first term, there is a theme for his second: fairness. Like a mantra at Charlotte, speakers argued for the middle class to have a "fair shot" and the rich to give their "fair share" as they carefully made it clear that inequality of outcome and income were tolerable so long as equality of opportunity was available. Massachusetts Senate candidate Elizabeth Warren received a huge cheer when she said: "People feel like the system is rigged against them. And here's the painful part: they're right. The system is rigged. Look around. Oil companies guzzle down billions in subsidies. Billionaires pay lower tax rates than their secretaries. Wall Street CEOs – the same ones who wrecked our economy and destroyed millions of jobs – still strut around Congress, no shame, demanding favours, and acting like we should thank them."

The second gift has been Romney: a wooden candidate whose personal wealth amounts to double the combined wealth of the last eight presidents going back to Richard Nixon and yet only pays 14% tax. For if Occupy Wall Street reframed the debate, then it also provided the basis to depict Romney as out-of-touch magnate with a tin ear for the travails of the common man. Most of the Democratic attacks over the summer, over the firings and outsourcings at Bain Capital – the firm that Romney once ran – and over demands for his tax records, or about his wealthy donors, fit that mould. This rich guy doesn't understand you or what you're going through, and now he wants to buy the election.

By contrast, Obama has put his less fortunate roots front and centre, running not on a narrative of racial breakthrough but of class mobility. "Barack knows the American dream because he's lived it," said Michelle Obama in her convention speech. "And he wants everyone in this country to have that same opportunity, no matter who we are, or where we're from, or what we look like, or who we love."

So this is the central strategy of the Obama campaign: not to lift people with lofty rhetoric, but hit them with a hard choice between him and Romney, and characterise it as a choice for either greater fairness or less; for the country to go backwards or forwards; someone who understands you or someone who doesn't. That tack seems to be paying off. Polls show that in swing states, where people are more likely to have seen the ads, they are twice as likely to see Romney's time at Bain as a reason to vote against him. Elsewhere the nation is evenly split.

In the crucial swing states of Ohio, Pennsylvania and Florida, where voters were asked the question: "Would you say that Mitt Romney cares about the needs and problems of people like you, or not?", most said "not". The poll showed that 54% of likely voters in Pennsylvania, 55% in Ohio, and 49% in Florida felt that Romney did not care about their problems. The Republican convention was dedicated in no small part turning that perception around. Arguably, it didn't work.

So while Obama is vulnerable, he is nonetheless ahead. In national polls his lead is narrow – within the margin of error – but has widened since the convention. In the swing states he is performing better. Last time around he took nine states that Republicans had won in 2004; New Mexico, Nevada, Colorado, Iowa, Indiana, Ohio, Virginia, North Carolina and Florida. But according to Real Clear Politics' average of polls, he leads in seven of them and trails in just two (Indiana and North Carolina). The New York Times' Nate Silver, a well-respected statistician, psephologist and author of the Five Thirty Eight blog, currently gives Obama a 75% chance of winning. And 59% of voters now believe Obama will win – although this is far more than will actually vote for him.

So the president has plenty of wriggle room as he seeks a path to victory this time around. He could lose the marquee states of Ohio and Florida, as well as the smaller states of New Hampshire, Iowa and Indiana, and still get his second term.

But it may not even come to that. Republicans were supposed to have picked much of the low hanging fruit clean by summer's end. But only in Indiana is Romney truly secure, while in North Carolina he is only now pulling away. Of the states Democrats won in 2004, only in Wisconsin is Obama really vulnerable. In some previously closely contested states, such as New Mexico and Pennsylvania, Romney is barely in contention. In the rest, Obama's lead is narrow (within the margin of error) but consistent and growing. A poll late last week showed Romney trailing Obama by at least five points in Florida, Virginia and Ohio. Although Romney wouldn't have to shift the needle very far to win, he'd have to shift a lot of needles in the same direction to stand a chance. Obama has precious little reason to be complacent, but every reason to be optimistic at this stage. His response to events like the riots across the Muslim world could still shift the trajectory, as could Republicans' cash advantage.

Obama supporters in New Hampshire. Can he fire up his base? Photograph: Saul Loeb/AFP/Getty Images

In a race where there will be few undecideds and a lot of cash sloshing around, Obama's challenge is to rally his base. This will be no simple task. Obama has considerable work to do on this front. Unlike four years ago, Republican voters are more likely to say it really matters who wins this election, to say they have given quite a lot of thought to the election and to have paid close attention to news about the election. Their base – white, wealthy, older people – is also more likely to turn out. The good news for the Democrats is that voters are more enthusiastic about their candidate. The percentage of people who said they strongly supported Obama was both higher than any Democratic candidate since 1988, including himself in 2008, and almost double those who strongly support Mitt Romney. The only candidate who's ever scored higher was George Bush in 2004.

Democrat supporters also appear more engaged. A survey of web and social media usage by Pew found that the Obama campaign posted nearly four times as much content as the Romney campaign on the web, was active on nearly twice as many platforms and prompted twice the number of shares, views and comments.

In 2008, Obama's victory was due, in no small part, to his ability to expand the electorate by attracting constituencies that had previously been under-represented at the polls – particularly young, black and Latino voters. He did not just win them by huge margins, but managed to amplify their electoral clout by motivating them to turn out in huge numbers.

But those are the very groups who have suffered most under his presidency. Unemployment for 18- to 19-year-olds is 23.5%; for those 20 to 24 it's lower at 12.9%, but still significantly higher than the national rate of 8.2%. Black unemployment is at 14.1% – a 10% increase on when he was inaugurated. While Hispanic unemployment has remained steady, Obama has deported more undocumented immigrants than any president since the 1950s.

Polls show Obama still holds a significant advantage among all three groups, attracting 89% of the black vote and 60% of both the Latino vote and the 18-to-29 age group. The issue is less whether those people will vote for him but how many will show up, since all three groups are less reliable voters. "We haven't seen much of the stimulus trickle down to our people here," Mark Allen, a Chicago-based community organiser who used to work alongside Obama, told the Washington Post. "I liked the community organizer Obama better than President Obama … Democrats say Barack has got 90% or whatever of the black vote wrapped up. What they don't tell you is it's 90% of those who actually come out and vote. What if it's 90% of just 30 or 40% who vote?"

In fact, the black turnout is the one part of his base that remains solid, but Democrats are less certain of Latinos and the young. In a base election, enthusiasm is key. Since black, Latino and young people live everywhere, there is not a single swing state where this is not an issue, and arguably only New Hampshire and Iowa – two of the whitest states in the country – where it is not key. This in no small part explains his decision to use the power of his office in June to halt the deportation of thousands of young undocumented immigrants – something he could have done at any time during his presidency. In an executive order he ruled that young immigrants who arrived in the US illegally before age 16 and spent at least five continuous years here would be allowed to stay and apply for work permits, providing they had no criminal history and met other criteria, such as graduating from high school or serving honorably in the military.

This time around, he is also seeking to make inroads among white women, many of whom are turned off by Republican views on abortion and contraception, pensioners, who may be nervous about Republican plans for Medicare (both of which he lost by 53-45 in 2008), and gay voters, buoyed by his support for gay marriage and motivated by Republican opposition to it. If there is any volatility in this race, it is not about the breadth of his support but the depth of it. National polls of registered voters may mostly show him with a narrow lead; but polls of likely voters often show him trailing.

The one thing that hasn't changed since 2008 is the Democrats' emphasis on the "ground game" – sending volunteers out to collect information, persuade and, ultimately, ferry people to the polls. One Washington Post survey showed 20% of registered voters had been contacted by the Obama campaign, compared to 13% who'd been contacted by Mitt Romney's campaign. And Democrats had been particularly effective at reaching their base. Forty-two percent of liberal Democrats said they'd been contacted, as well as 24% non-whites and 31% of the people who voted for Obama last time.

"This is light years ahead of where we were in 2008," said Obama's campaign manager, Jim Messina, during a forum in Charlotte. "We are going to make 2008, on the ground, look like Jurassic Park." They plan to knock on more than twice as many doors and register twice as many voters as they did last time. In North Carolina they have twice as many field offices as the Romney campaign. In Ohio they have three times as many.

If Obama were only running against Romney, his victory would be all but assured by this stage. But he isn't. He's running against the economy, and on his promise to be a transformative president in tough times. To the extent that a second-term election is a referendum on the incumbent, Obama is losing. Too many voters think he's not done enough. To the degree, however, to which any election is a choice between at least two candidates, he is winning. Enough voters feel he is better than his opponent. For now, at least, he is not so much the change they can believe in as the change agent they most believe in. And while the semantic difference is minimal, the rhetorical difference could hardly be greater.