On 15 January 1998, what would have been Martin Luther King’s 69th birthday, James Farmer was awarded the presidential medal of freedom in the White House’s East Room. “He has never sought the limelight,” said the then president, Bill Clinton. “And until today, I frankly think he’s never got the credit he deserves. His long overdue recognition has come to pass.”

Farmer, who ran the Congress of Racial Equality and led the Freedom Rides through the segregated south in 1961, was by that time blind, diabetic and a double amputee. He died the following year. When I spoke to him a few months after the ceremony, he said it was the best day of his life. “It was just like the old days. But this time, I felt like I was finally being vindicated; that the years of invisibility were over.”

And what, I asked, had rendered him invisible? Having a white wife, he said, had been an issue. But the main reason, he believed, was that the US had only enough room to remember one civil rights leader. “I was in the shadow of Martin Luther King,” he told me. “And that was quite a big shadow. It was King who made the ‘I have a dream’ speech. And King was assassinated – and that always enlarges a person’s image.”

Half a century after King’s assassination, there is value in reflecting on that shadow. The angle at which King’s body of work caught the light after he was killed tells us a great deal about how a black radical preacher came to be so big and what else may be hidden in the darkness.

This week, the US will indulge in an orgy of self-congratulation, selectively misrepresenting King’s life and work, as if rebelling against the American establishment was, in fact, what the establishment has always encouraged. They will cite the “dream” speech as if it were his only one – and the line about wanting his children to be “judged not by the colour of their skin but the content of their character” as if it were the only line in it.

In so doing they will wilfully and brazenly omit the fact that before his death in 1968, King was well on the way to becoming a pariah. In 1966, twice as many Americans had an unfavourable opinion of him as a favourable one. Life magazine branded his anti-Vietnam war speech at the Riverside church, delivered exactly a year before his assassination, as “demagogic slander”, and “a script for Radio Hanoi”. Just a week before he was killed, he attended a demonstration in Memphis in support of striking garbage workers. The protest turned violent and police responded with batons and teargas, shooting a 16-year-old boy dead. The press and the political class rounded on King. The New York Times said the events were “a powerful embarrassment” to him. A column in the Dallas Morning News called King “the headline-hunting high priest of nonviolent violence” whose “road show” in Memphis was “like a torchbearer sprinting into a powder-house”. The Providence Sunday Journal called him “reckless and irresponsible”. He was back in Memphis supporting the strike when he was killed.

A 1967 protest about the use of federal soldiers during riots in New Jersey. Photograph: Three Lions/Getty Images

This was the last time King received national coverage when he was alive, and so he died a polarising and increasingly isolated figure. Just six days after his death, the Virginia congressman William Tuck blamed King for his own murder, telling the House of Representatives that King “fomented discord and strife between the races … He who sows the seed of sin shall reap and harvest a whirlwind of evil.”

But in the intervening decades, the mud slung at him has been cleaned off and his legacy shined to make him resemble a national treasure. In the two years before his death, he did not appear in the Top 10 of Gallup’s poll of most admired men of the year. In 1999, a Gallup poll of the most admired people of the century placed him second behind Mother Teresa. In 2011, King’s memorial was opened on the National Mall in Washington DC, with a 30ft statue sitting on four acres of prime historic real estate: 91% of Americans (including 89% of white people) approved. Even Donald Trump has thus far refrained from besmirching his legacy, hailing just a few months ago King’s “legacy of equality, justice and freedom”.

The process by which King went from ignominy to icon was not simply a matter of time and tide eroding ill feelings and painful memories. “History” does not objectively sift through radical leaders, pick out the best on their merits and then dedicate them faithfully to public memory. It commits itself to the task with great prejudice and fickle appreciation in a manner that tells us as much about historians and their times as the leaders themselves.

“The facts of history never come to us pure,” wrote EH Carr in his seminal essay The Historian and His Facts. “Since they do not and cannot exist in pure form: they are always refracted through the mind of the recorder … History means interpretation … It is the historian who has decided for his own reasons that Caesar’s crossing of that petty stream, the Rubicon, is a fact of history, whereas the crossing of the Rubicon by millions of other people before or since interests nobody at all.”

Today’s understanding of King is the result of a protracted struggle and strategic reckoning involving an ongoing national negotiation about how to understand the country’s racial narrative. White America did not make the journey towards formal equality willingly. A month before the March on Washington in 1963, 54% of white Americans thought the Kennedy administration “was pushing racial integration too fast”. A few months later, 59% of northern white Americans and 78% of southern white Americans disapproved “of actions Negroes have taken to obtain civil rights”. That same year, 78% of white southern parents and 33% of white northern parents objected to sending their children to a school in which half the students were black. According to Gallup, it was not until 1995 that a majority of white US citizens approved of marriage between black and white people.

To discount King would be to dismiss the most prominent and popular proponent of civil rights. That in turn would demand some other explanation as to how the US shed the stigma of segregation and imagined itself as a modern, nonracial democracy. For while the means by which codified segregation came to an end – mass marches, civil disobedience, grassroots activism – was not consensual, the country did reach a consensus that it had to end. But there was no plausible account for how they travelled from Rosa Parks to Barack Obama that does not have King front and centre, even if the gap between black and white unemployment is roughly the same now as it was in 1963, southern schools are resegregating and the wealth gap is widening.

King being arrested in Birmingham, Alabama, in 1963. Photograph: AP

So, white America came to embrace King in the same way that white South Africans came to embrace Nelson Mandela: grudgingly and gratefully, retrospectively, selectively, without grace or guile. Because, by the time they realised their hatred of him was spent and futile, he had created a world in which loving him was in their own self-interest. Because, in short, they had no choice.

“Our country has chosen the easier way to work with King,” the late Vincent Harding, who wrote a draft of King’s Riverside speech, told me. “They are aware that something very powerful was connected to him and he was connected to it.”

It has not been a straightforward journey for black America either. King was always popular with black Americans, though not always with black political leaders – younger activists mockingly referred to him as “de lawd” because he was so grand, while his contemporaries criticised him for parachuting in on conflicts to great media attention. But the victories for civil rights soon came up against the legacy of several centuries of oppression and the realities of capitalism. In short, what does racial equality look like in a country where economic inequality is deeply ingrained into the system – what is the value of being able to eat in a restaurant of your choice if you can’t afford what’s on the menu?

At a meeting in Chicago in 1965, King was shaken after he was booed by young black men in the crowd: “I went home that night with an ugly feeling, selfishly I thought of my sufferings and sacrifices over the last 12 years,” he recalled. “Why should they boo one so close to them? But as I lay awake thinking, I finally came to myself and I could not for the life of me have less than patience and understanding for those young men. For 12 years, I and others like me have held out radiant promises of progress, I had preached to them about my dream … I had urged them to have faith in America and in white society. Their hopes had soared. They were now booing me because they felt that we were unable to deliver on our promises. They were booing because we had urged them to have faith in people who had too often proved to be unfaithful. They were now hostile because they were watching the dream they had so readily accepted turn into a frustrating nightmare.”

So the King America has chosen to remember stands at Lincoln’s feet talking of a dream “deeply rooted in the American dream”. But by the long, hot summer of 1967, which saw riots in Newark, Cincinnati and Buffalo, and tanks rolling down the streets of Detroit, King had, in the midst of the cold war, moved on to questioning capitalism. “We must honestly face the fact that the movement must address itself to the question of restructuring the whole of American society,” he said in August 1967. “There are 40 million poor people here, and one day we must ask the question, ‘Why are there 40 million poor people in America?’ And when you begin to ask that question, you are raising a question about the economic system, about a broader distribution of wealth. When you ask that question, you begin to question the capitalistic economy … when you deal with this, you begin to ask the question: ‘Who owns the oil?’ You begin to ask the question: ‘Who owns the iron ore?’ You begin to ask the question: ‘Why is it that people have to pay water bills in a world that’s two-thirds water?’”

In 2002, when I first interviewed the late poet and author, Maya Angelou, about a volume of her memoir which covered the assassinations of King and Malcolm X, I asked what she thought they would be focusing on had they lived. My normally loquacious interviewee sat silent before shaking her head and releasing a long, helpless breath. “I can’t,” she said. “I can’t. So many things have happened since they were both assassinated. The world has changed so dramatically.”

Reputations forged in revolutionary periods can rarely be sustained through calmer times. In moments of social turmoil and sharpened conflict, the stage yields to the declarative, decisive, emphatic and bold; to those who can rally their own side and face down their tormentors. Revolutionary moments favour not just the single-minded but the reckless, which, in King’s case, meant a man who was prepared to wear his funeral clothes to work.

I once asked Jack O’Dell, one of King’s long-term aides, why King had delivered his speech about Vietnam when he knew it would ruin his relationship with the White House and cost the movement a lot of support and funds. “He had the Nobel prize,” said O’Dell, “and he didn’t know how long he was going to live. He wasn’t but 39, but he wasn’t going to live much longer, and that meant he didn’t have but maybe a few more speeches to give. So he had to say what he was going to say.”

But once the conditions that make those periods possible evolve, so do the leadership skills necessary for the new moment. Those who are most effective at the barricades aren’t necessarily best equipped for the boardroom, which is why the transition from guerilla to government is so fraught for so many movements. That evolution never stops.

So Jesse Jackson, who was with King in Memphis the day he died, reinvented himself as an electoral contender during the 80s, channelling the spirit of the civil rights movement into a broad-based coalition embracing unions, feminists, gay rights activists and environmentalists that posed a challenge to the Democratic party. But, 20 years later, he was heckled by black protesters against police brutality in Ferguson, Missouri. In her memoir, When They Call You a Terrorist, one of the Black Lives Matter founders, Patrisse Khan-Cullors, berates those “black pastors and then the first black president [who] preached [about personal responsibility] more than they preached a commitment to collective responsibility”.

And many turn the pages of history quicker than they can read, let alone understand them. John Lewis, the sole living speaker from the March on Washington in 1963, questioned Trump’s legitimacy given the alleged involvement of Russia in the election. Lewis, who was bludgeoned by bigots in Birmingham, Alabama in 1961 and by police in Selma in 1965, was criticised by Trump on Twitter in January last year for being “all talk, talk, talk – no action, no results”.

So in life, King’s one-time contemporaries struggle, as he did, with a white America that is dismissive and a black America that demands more than their movements can deliver. In death, the struggle is to ensure that King’s legacy isn’t eviscerated of all militancy so that it can be repurposed as one more illustration of the American establishment’s God-given ability to produce the antidote to it’s own poison.

This contradiction was most blatant in February, during the Super Bowl, when Ram trucks used a recording of a sermon King delivered about the value of service to sell its product – “Built to serve”.

The insult to King’s memory could not have been more absolute. In another part of the sermon Ram used, King literally tells the congregation not to be fooled into spending more money than necessary on cars by sharp advertisers. “These gentlemen of massive verbal persuasion,” he says, “have a way of saying things to you that kind of gets you into buying … In order to make your neighbours envious, you must drive this type of car … And before you know it you’re just buying that stuff.” In a week that will see almost every stratum of American society – the society that voted for Trump – mourn King’s passing and hail his contributions, this may well be the most important message of all.

This article includes material excerpted from Gary Younge’s book, The Speech: The Story Behind Dr Martin Luther King Jr’s Dream