This is an edited extract of the 2016 A. N. Smith Lecture in Journalism, delivered by Emily Bell, the founding director of Columbia University’s Tow Centre for Digital Journalism, at the University of Melbourne on March 15, 2017.

The 2016 US presidential election tells us a great deal about the current state of the news media and, more importantly, about the information environment we are operating within. Some of it is shocking, some is deeply worrying, and some of it is hopeful.

There are four key things Donald Trump’s election tells us about the state of journalism today.

The new fusion of media, power and technology

It is the early morning of Sunday, March 5. All over the east coast of America, journalists’ phones vibrate with alerts. So it begins: the president is awake, and he is angry.

It is as well, with the terrible decline in the popularity of The Apprentice, that we have another mesmerising show to keep us on our toes.

Imagine being an American political journalist. Every Saturday and Sunday at 6AM or 7AM, your phone buzzes with a message from the president.

Barack Obama “had my wires tapped in Trump Tower”. Quite a serious allegation. Presidents have been impeached for less. Press Secretary Sean Spicer was at pains to point out the next week that this is not literally what Trump had meant – it was a broader referral to the activities of agencies and surveillance during the campaign.

Welcome to 2017 in the United States of America, where we can experience government in real time, sometimes even before the people in power find out about it themselves.

As journalists who cover Trump have told me, the president, like his audience, reads newspapers and watches cable news. This is why he tweets early in the morning, often about stories that broke on social media the previous day. He opens the “failing” New York Times and, provoked by their “fake news”, he is off.

Trump might not spend much time on social media but he has an acute understanding of how virality in media works, and what the dynamics are that are needed to activate an online following to amplify your message.

How to cover the president is an abiding press topic, because he is unlike any president most have ever seen. Invitations to press conferences where journalists were ignored or insulted. Press huddles that suddenly elevated outlets like OANN or Breitbart above the “failing” New York Times and the “fake news” Washington Post.

In elevating Breitbart’s Steve Bannon to be his chief strategist, Trump has consolidated the idea of putting media presence at the heart of his administration.

There were many media commentaries suggesting that journalists should boycott the press room at the White House. Another extended hand-wringing session took place around whether or not to cover Trump’s tweets at all – again, were the media being played? Was access journalism getting in the way of real stories? And there were questions on how to deal with Trump’s apparent lack of interest in whether something was true or not.

What has happened is that Trump’s Twitter feed and the White House press room have become the live rails of this administration. If you start to think about Trump and Bannon as a media organisation run from the Oval Office, it makes sense that the PR channels of the press room and a Twitter feed with 25 million followers are actually now live policy theatre.

The tweeting, the press conferences and the rallies are confusing for us because they feel like smokescreens. Even more unsettling is the notion that they are not smokescreens at all, but they are the actual presidency.

Political messaging on social media has come of age in a powerful way. In the election of Rodrigo Duterte in the Philippines, we saw similar patterns of deployment as we did in the Trump campaign: tight circles of hyperactive fans and bots on social media that co-ordinate to tweet hashtags signalling the campaign’s most important messages in a repetitious cycle: #TrumpWon, #CrookedHillary, #HillarysHealth.

These are commonly used tactics, and they are bipartisan. Hillary Clinton’s campaign was just as active in similar ways.

But, for Trump, the use of real-time communication was not simply an election tactic: it is his modus operandi. Effectively, the substance of the US government is being shaped by a social media app.

Online media reinforce homogeneity of view

According to the Pew Research Centre in the US, more than 70% of the population now has a smartphone. Of that universe, more than 60% get news through social media. Around 45% of US adults say they get news from Facebook. This is an enormous amount of power concentrated in one newsfeed algorithm.

For more than a decade, 1,000 flowers bloomed on the open web, and 1,000 tabs opened on each desktop.

This diversity is threatened with the commercialised, mobile social web. Smartphones and social media, which work in lock-step to focus our attention on the smaller screen, have been a great rebundling of news services – and a great rebundling of all services.

Facebook, I once observed, was swallowing journalism. But it is also swallowing everything else too.

As the user base has grown from millions to hundreds of millions to billions, the sorting algorithms target us not to show us everything – that would be unmanageable and absurd – but to show us each our own heavily personalised version of the world.

The ubiquity of social media and the way its business model works, targeting us with more of what we like, is an open invitation to stay in our lane – in our interests, our geographies, our views, our media and our lives.

The really efficient thing about social media is we don’t have to even try to do that ourselves anymore. The mysterious algorithmic underpinnings of Google and Facebook do it for us, and we don’t even notice. Until we miss something that happened in someone else’s lane. For liberal America, Trump happened in someone else’s lane.

Breitbart, more than any other news site, represents the noisy voice of the far right. Two weeks ago, the Berkman Klein Center for Internet and Society at Harvard released a paper that looked at how a large, fast-growing, right-wing news ecosystem had grown rapidly within Facebook in a relatively short time. Breitbart is the epicentre of that particular echo chamber.

The researchers mapped how that particular ecosystem works. They noted that most of the new hyper-partisan right-wing websites saw few links with the mainstream media, that the readership on the right was more isolated, and that the sites were very efficient in recycling the same themes – Trump’s anti-immigration stance, and Clinton’s emails.

As the campaign wore on the Harvard study notes the attacks were routinely targeted at both Clinton and the “mainstream media”. These messages were also repeated often by Trump. The campaign for him was arguably never about policy development: it was about a ratings-driven approach to winning.

The study’s authors say:

Use of disinformation by partisan media sources is neither new nor limited to the right wing, but the insulation of the partisan right-wing media from traditional journalistic media sources, and the vehemence of its attacks on journalism in common cause with a similarly outspoken president, is new and distinctive.

One of the remarkable things about the 2016 election was how the two worlds of polarised opinion could co-exist without any central arbitration by more moderate “connectors”.

The independent press should have a role here to moderate and foster argument from the point of view of trying to reach a broad consensus. But there is only a small incentive left to do that.

Watching a Trump press conference on Facebook Live video streamed from the Washington Post’s page showed only glowering angry emoji faces. The same stream, viewed from Fox News’ page, had only smiles.

I have wondered more than once during this election cycle whether the American media landscape is particularly badly affected by the encroachment and rise of newer, less-well-known and more-partisan forces because it lacks an equivalent of the ABC or BBC.

Obsession with ‘fake news’ obscures the real problems

There is something mesmerising about watching what happens when people are able to continually lie without facing the consequences.

After his initial Obama wiretapping accusations, Trump officials adjusted their body language about the reasons to believe, or not believe, that this “wiretap” had happened.

The story seems to have originated on right-wing talk radio and was picked up by other right-wing media, and then repeated by Trump on Twitter. If this is not true, though, what are the consequences?

Does a lack of truth matter? The tweet is not about the truth: it is about enriching a distracting narrative.

As a profession and a field we have to acknowledge the role we have played over many years in creating a commercial media environment that places higher priority on readership, ratings and reach than on the absolute integrity of information.

The open web was meant to make this better. It was initially a great big engine for correcting and contesting what is published. Instead, on balance, the form of the web we have at the moment enables bad journalism as much as, or maybe even more than, it helps good journalism.

The key to this is in the workings of the advertising market. It is increasingly automated, and decreasingly regulated.

In a digital microtargeted environment, ads are sold not against the integrity of the publication hosting them, but on the value of the person seeing them. Why pay the Wall Street Journal’s ad rates when you can buy one of their desirable readers a couple of sites or pages away?

And how to engage those readers? Well, good jokes and sensational content works better than nuance and complexity.

A combination of human nature, commercial marketplaces and sophisticated large-scale technology has combined to produce almost-perfect conditions for the proliferation of lowest-common-denominator material. The “fake news” epidemic is not new either, but the electrifying possibility it might have contributed to upending democracy has pushed it to the forefront of the debate.

I am not a fan of the phrase “fake news”. The term in Trump’s hands can mean “news you don’t need to pay attention to” or “news I don’t like”. We ought to be calling propaganda what it is, and calling misinformation and lies what they are too.

Buzzfeed media editor Craig Silverman has been into this issue for years. In 2015 he wrote a white paper for the Tow Centre entitled Lies, Damn Lies and Viral Content. It looked at the growth of precisely this epidemic in rumours that circulate through social media at lightning speed and are proliferated and amplified by mainstream news outlets at a much higher rate than they are ever corrected.

It was in the course of doing this work that Silverman started to notice the same patterns confirmed by the Harvard research – notably that the hyper-partisan websites, particularly of the right, were the powerhouses for a particular type of disinformation.

The incentives for feeding made-up stories or maybe even sentiments and stories into this cycle are twofold. First, they are political. Second, they are financial.

In the election these fabricated stories soared in popularity, outperforming the real news stories on Facebook in the closing stages of the campaign.

These were both propagandistic and opportunistic.

“Fake news” was undoubtedly put out by both sides during the election to benefit their preferred candidate. But another type of fakery was manufactured, quite legally, by people who can exploit the attention market for great profit. Someone who actually makes fake news (or, rather, used to) told me he could make US$10,000 from a single post.

In the future, we will see both the automation and more authentic fabrication of material. It is not clear that the platform companies are winning the war with faked propagandistic messages. But it is clear that they have been too relaxed about the type of material that circulates, whether for political or economic advantage.

If the advertising model rewards popularity and shareability – regardless of originality, value and quality – then it is little wonder that it provides a living for a Macedonian teenager but not enough to support core reporting functions in local newsrooms.

Facebook founder Mark Zuckerberg has said he does not want to be the “arbiter of truth” – which is lucky, because at the moment that is a distant aspiration. Perhaps a more achievable aspiration is not to be the enemy of truth either.

There is a cultural confusion within technology companies about how they execute both their ideological and market positions to encourage the maximum participation – but not to edit their platforms.

The major platforms, and Facebook in particular, were unsettled by the result of the 2016 election and the role they might or might not have played in Trump’s surprising rise. Why else would Zuckerberg write a 6,000-word manifesto about the issues facing global citizens and what Facebook could do to fix them?

The social web is visual; it rewards jokes and comments and easily digestible commentary. It rewards feelings and emotions. It rewards intensity of usage and engagement. It does not really care about veracity or verifiability.

Some of this is not new at all. Tabloid newspapers have always had more readership than broadsheets; cartoonists are more famous than op-ed writers. But the uneven distribution of attention on the web, and the algorithmic response to that – broadly promoting more of the same – shows again that the breaking of the distribution monopoly of old media has been replaced by another kind of monopoly, a monopoly of time, or attention.

Fake news has become a meaningless and rather dangerous phrase. But the problem of feeling unsure of what to believe and what not to believe, the obliteration of credible brands and the squeezing of all types of content into the same undelineated window, is very real.

Mark Zuckerberg says he does not want Facebook to be ‘the arbiter of truth’.Reuters/Stephen Lam

Journalism matters

In Western democracies we have become used to the luxury of being sceptical and dismissive of the importance of a free press. Now in the White House Press Room, on Twitter and Facebook, in the feed of the president of the United States of America, that dismissiveness has turned to an open hostility.

I know why phrases like “post-truth societies” or “alternative facts” or “fake news” have taken hold as a result of the election. It is important, though, to be able to separate media theory from reality. We are not in fact living in a world where facts and truth don’t exist anymore.

People who care about democracy recognise this, and the US has seen what is known as the “Trump bump” for news organisations with subscription or membership models. The “failing” New York Times has seen almost 300,000 new subscribers join in the last quarter of 2016 – more additional subscribers than the organisation managed in the previous year.

The head of digital at another national subscription-based news organisation noted that:

Every time Trump tweets about how terrible we are, another 10,000 people give us money. It’s incredible.

Non-profit investigative unit ProPublica has seen enough voluntary contributions coming in to enable it to open another office. The Committee to Protect Journalists, a great organisation that does the hardest work in defending and protecting journalists around the world, was invoked by Meryl Streep, that “over-rated actress”, on stage at the Golden Globes.

I was recently at a marketing conference full of marketing executives talking on stage to two Washington Post executives who were given a spontaneous ovation – by advertisers.

Despite the rocky professional outlook, there is no decline in the number of people applying to Columbia Journalism School – quite the opposite.

And, underneath this, we have seen some really remarkable reporting and analysis from the campaign trail and now from inside the administration. Leakers are posting material through Secure Drop to newsrooms at unprecendented rates.

The advent of a president who calls the press “the enemy of the people” has galvanised news organisations and handed them a mandate.

And the light the election campaign has shone on what an information environment can become, without regulation and without a hierarchy that reflects civic values, I think has rejuvenated the case for a strong and independent press.

Even Zuckerberg recognises the importance of journalism. On President’s Day weekend, he posed outside the offices of the Selma Times-Journal with his wife, Priscilla Chan, and thanked journalists for their work. Via a Facebook post, of course.

In his 6,000 word manifesto he wrote about what Facebook might do to support journalism more:

There is more we must do to support the news industry to make sure this vital social function is sustainable – from growing local news, to developing formats best suited to mobile devices, to improving the range of business models news organisations rely on.

He did not mention a significant transfer of wealth – but maybe that is coming.

What we have seen in the 2016 election cycle though is very clarifying for journalism. We have seen that the information ecosystem has grown in ways that work against the interests of civic society and good journalism.

The functions of journalism – from the packaging and distribution, to the audiences and branding, to the data collection, and crucially the monetisation – have all been subsumed by much larger systems of power and wealth.

Until very recently, technology platforms were ambivalent or even hostile to the idea that they might bear some responsibility for creating a better public sphere. The election cycle of 2016 has shone a light on that too. There is no such thing as algorithmic neutrality. Platforms and technologies have values, and if they carry consequences, intended and accidental.

Recently, a smart, local start-up in upstate New York, the Watershed Post, announced it could not do what it was doing anymore. A very technically literate two-person team had set up the Watershed Post as a new model for local journalism. Founder Lissa Harris wrote about why they could no longer carry on:

The titans of the web have huge and increasing reach, even in our rural communities. They have sophisticated tools for targeting likely customers by geography and demographics. They have products that a business owner can buy for $5 with a few clicks of a mouse, products that require no human time investment on the other end for design or sales or customer support.

What they don’t have is reporters.

Journalism matters, but the institutions that support and contain reporting are only healthy if they have subscribers, or vast scale, or another source of revenue. In the US, this increasingly means philanthropy, or a return to the wealthy individual sinking hundreds of millions into an uncertain future.

And the promise of the open web – that it would support all type of new journalistic institutions – is unfulfilled.

Reporting, unlike memes and jokes and native advertising, does not scale well on the privatised social mobile web. This is not the fault of one set of people; I don’t believe that the founders of platforms and search companies wanted to destabilise functions that are civically important but financially insecure. And I don’t believe the generation of creative, technically gifted journalists who are struggling with this necessarily did anything wrong either.

The problem is more that the speed of the emerging landscape for media has been so quick, and largely illegible, so free of regulation in nearly all aspects, that rather like financial deregulation before it, we haven’t been able to really grasp the problem until it is almost too late.

I say “almost” because, as an eternal optimist, I think we have an opportunity to make the right interventions to press for systems that favour sustainable journalism. But we have to be organised, and we have to do it now.

We need better collective action in understanding the complexity of the problems, and we need institutions that will not be buffeted by the markets to help work these problems out in the long term.

We also need the attention, wealth and influence of technology companies.

I have been dismayed as a bystander to see how institutions like the BBC in the UK and the ABC to some extent in Australia are not apparently part of the central conversation for this resettlement for journalism. Their own issues are too often framed in terms of the market and not often enough in terms of civic need and what we might require independent media institutions to do to protect democracy.

Too often we have given into the Silicon Valley narrative that old institutions are inevitably going to perish.

The “free market of information” on the web erases the necessity for old-fashioned public interventions, doesn’t it? The market will fix everything, won’t it? I could not disagree more.

Leadership in public-service journalism institutions is at a critical moment where it can redefine its role in relationship to this new landscape; where it can make a strong case for the support of reporting and innovation that can endure separately from the alternate systems of tech power.

Public media is not the only solution, but it is an important element in figuring out how we manage our way through a complicated and rapidly changing commercial converged marketplace.

I was recently at a rather curious gathering in the UK countryside – between NGOs, government, technology companies and journalists – to talk about the crisis in news and information. I can’t do better than repeat the words of one of the attendees:

We should be thanking Donald Trump, because without him we would not be having the long overdue conversation we need about what kind of news environment we want.