The scientific revolution in the 16th century was one of the biggest societal shifts in human history, driven by the discovery of new and better methods of figuring out who was right and who was wrong.

Julia Galef – a well-known writer and researcher focused on improving human judgment, especially about high stakes questions – believes that if we could develop new techniques to resolve disagreements, predict the future and make sound decisions together, we could again dramatically improve the world. We brought her in to talk about her ideas.

What natural disaster is most likely to kill more than 10 million human beings in the next 20 years?

Terrorism? Famine? An asteroid?

Actually it’s probably a pandemic: a deadly new disease that spreads out of control. We’ve recently seen the risks with Ebola and swine flu, but they pale in comparison to the Spanish flu which killed 3% of the world’s population in 1918 to 1920. If a pandemic of that scale happened again today, 200 million would die.

Looking back further, the Black Death killed 30 to 60% of Europe’s population, which would today be two to four billion globally.

The world is woefully unprepared to deal with new diseases. Many countries have weak or non-existent health services. Diseases can spread worldwide in days due to air travel. And international efforts to limit the spread of new diseases are slow, if they happen at all.

Even more worryingly, scientific advances are making it easier to create diseases much worse than anything nature could throw at us – whether by accident or deliberately.

In this in-depth interview I speak to Howie Lempel, who spent years studying pandemic preparedness for the Open Philanthropy Project. We spend the first 20 minutes covering his work as a foundation grant-maker, then discuss how bad the pandemic problem is, why it’s probably getting worse, and what can be done about it. In the second half of the interview we go through what you personally could study and where you could work to tackle one of the worst threats facing humanity.

OpenAI’s Universe, a software platform for training AIs to play computer games.

Just two years ago OpenAI didn’t exist. It’s now among the most elite groups of machine learning researchers. They’re trying to make an AI that’s smarter than humans and have $1b at their disposal.

Even stranger for a Silicon Valley start-up, it’s not a business, but rather a non-profit founded by Elon Musk and Sam Altman among others, to ensure the benefits of AI are distributed broadly to all of society.

I did a long interview with one of its first machine learning researchers, Dr Dario Amodei, to learn about:

OpenAI’s latest plans and research progress.

His paper Concrete Problems in AI Safety, which outlines five specific ways machine learning algorithms can act in dangerous ways their designers don’t intend – something OpenAI has to work to avoid.

How listeners can best go about pursuing a career in machine learning and AI development themselves.

We suggest subscribing, so you can listen at leisure on your phone, speed up the conversation if you like, and get notified about future episodes. You can subscribe by searching ‘80,000 Hours’ wherever you get your podcasts (RSS, SoundCloud, iTunes, Stitcher).

If a smarter-than-human AI system were developed, who would decide when it was safe to deploy? How can we discourage organisations from deploying such a technology prematurely to avoid being beaten to the post by a competitor? Should we expect the world’s top militaries to try to use AI systems for strategic advantage – and if so, do we need an international treaty to prevent an arms race?

Questions like this are the domain of AI policy experts.

We recently launched a detailed guide to pursuing careers in AI policy and strategy, put together by Miles Brundage at the University of Oxford’s Future of Humanity Institute.

I interviewed Miles to ask remaining questions I had after he finished his career guide. We discuss the main career paths; what to study; where to apply; how to get started; what topics are most in need of research; and what progress has been made in the field so far.

One way that people can have a social impact with their career is to donate money to effective charities. We mention this path in our career guide, suggesting that people donate to evidence-backed charities such as the Against Malaria Foundation, which is estimated by GiveWell to save the lives of children in the developing world for around $7,500.

Alyssa Vance told me that many people may see this as highly ineffective relatively to their optimistic expectations about how much it costs to improve the lives of people in people. I thought the reverse would be true – folks would be skeptical that charities in the developing world were effective at all. Fortunately Amazon Mechanical Turk makes it straightforward to survey public opinion at a low cost, so there was no need for us to sit around speculating. I suggested a survey on this question to someone in the effective altruism community with a lot of experience using Mechanical Turk – Spencer Greenberg of Clearer Thinking – and he went ahead and conducted one in just a few hours.

You can work through the survey people took yourself here and we’ve put the data and some details about the method in a footnote. The results clearly vindicated Alyssa:

It turns out that most Americans believe a child can be prevented from dying of preventable diseases for very little –

People in the effective altruism community often refer to the global income distribution to make various points:

The richest people in the world are many times richer than the poor.

People earning professional salaries in countries like the US are usually in the top 5% of global earnings and fairly often in the top 1%. This gives them a disproportionate ability to improve the world.

Many people in the world live in serious absolute poverty, surviving on as little as one hundredth the income of the upper-middle class in the US.

Measuring the global income distribution is very difficult and experts who attempt to do so end up with different results. However, these core points are supported by every attempt to measure the global income distribution that we’ve seen so far.

The rest of this post will discuss the global income distribution data we’ve referred to, the uncertainty inherent in that data, and why we believe our bottom lines hold up anyway.

Will MacAskill had a striking illustration of global individual income distribution in his book Doing Good Better, that has ended up in many other articles online, including our own career guide:

The data in this graph was put together back in 2012 using an approach suggested by Branko Milanovic,

When we first speak to people interested in doing good with their careers, they often say they want to get involved in education in the US or the UK. This could mean donating to a school, doing education policy work, or becoming a teacher.

However, we haven’t prioritised careers in education at 80,000 Hours. We don’t dispute that education is a highly important problem – a more educated population could enable us to solve many other global challenges, as well as yield major economic benefits. The problem is that it doesn’t seem to be very easy to solve or neglected (important elements of our problem framework). So, it looks harder to have a large impact in education compared to many other areas. In the rest of this post, we’ll give five reasons why.

The following isn’t the result of in-depth research; it’s just meant to explain why we’ve deprioritised education so far. Our views could easily change. Note that in this post we’re not discussing education in the developing world.

1. It’s harder to help people in the US or UK

Everyone in the US or UK is rich by global standards: the poorest 5% of Americans are richer than the richest 5% of Indians (and that’s adjusted for the difference in purchasing power, see an explanation and the full data).

Over the past three years, we’ve grown almost 36-fold, more than tripling each year. This is measured in terms of our key metric – the number of impact-adjusted significant plan changes each month. At the same time, our budget has only increased 27% per year.

Given this success, we think it’s time to take 80,000 Hours to the next level of funding.

Over the next few weeks, we’ll be preparing our full annual review and fundraising documents, but here’s a preview.

Overall, the 2017 target is to triple, measured in terms of impact-adjusted significant plan changes per month (which will mean over 3,000 over the year). We’ll do this by continuing to improve the advice, and starting to scale up marketing, with the aim of becoming the default source of career advice for talented, socially-motivated graduates.

Concretely, here’s some priorities we could pursue:

Dramatically improve the career reviews and problem profiles, so we have in-depth profiles of all the best options. This will help our existing users make better changes, and bring in more traffic.

Upgrading – develop mentors and specialist content for the most high-potential users, such as those who want to work on AI risk, policy, EA orgs and so on. We now have a large base of engaged users (1300+ through the workshop, 80,000+ on newsletter), so there’s a lot of follow-up we could do to get more valuable plan changes from them.

The following are some reflections on what it’s like to work at GiveWell written by one of our readers. We’re posting their thoughts because we’ve written about GiveWell as a high-impact career in the past, and are keen to share more information about it. The opinions below, however, may not reflect our views.

I worked at GiveWell from August 2014 to May 2016. This piece is a reflection on my time there, on things I think GiveWell does well as an employer, on things I think it could do better, and why I decided to leave.

I envision two functions for this piece: (1) as an exercise to help me process my time at GiveWell, and (2) as a resource for people considering working at GiveWell. When I was considering taking a job at GiveWell, I found Nick Beckstead’s reflection on his internship at GiveWell to be very helpful. Outside of Nick’s piece, there isn’t very much substantive information available about working at GiveWell. Many people consider employment at GiveWell; I hope some of those people find this reflection to be useful.

Some background

I learned about GiveWell in Spring 2014, after reading Peter Singer’s Famine, Affluence, and Morality in a college ethics class and encountering related topics on the internet. By the time I took the ethics class, I knew that I did not want to go to graduate school immediately after my undergraduate, but I was very taken by academic ethics and wanted to continue serious thinking about the topic.

Local laws often prohibit the construction of dense new housing, which drives up prices, especially in a few large high-wage urban areas. The increased prices transfer wealth from renters to landowners and push people away from centres of economic activity, which reduces their ability to get a job or earn higher wages, likely by a very large amount.

An opportunity to tackle the problem which nobody has yet taken is to start a non-profit or lobbying body to advocate for more housing construction in key urban areas and states. Another option would be to try to shift zoning decisions from local to state governments, where they are less likely to be determined by narrow local interests, especially existing land-owners who benefit from higher property prices.

In the profile we cover:

The main reasons for and against thinking that working on land use reform is among the best uses of your time.

How to use your career to make housing in prospering cities more accessible to ordinary people.

There is a small but non-negligible chance that unmitigated greenhouse emissions will lead to very large increases in global temperatures, which would likely have catastrophic consequences for life on Earth.

Though the chance of catastrophic outcomes is relatively low, the degree of harm that would result from large temperature increases is very high, meaning that the expected value of working on this problem may also be very high.

Options for working on this problem include academic research into the extreme risks of climate change or whether they might be mitigated by geoengineering. One can also advocate for reduced greenhouse emissions through careers in politics, think-tanks or journalism, and work on developing lower emissions technology as an engineer or scientist.

In the profile we cover:

The main reasons for and against thinking that the ‘tail risks’ of climate change are a highly pressing problem to work on.

How climate change scores on our assessment rubric for ranking the biggest problems in the world

Many experts believe that there is a significant chance we’ll create artificially intelligent machines with abilities surpassing those of humans – superintelligence – sometime during this century. These advances could lead to extremely positive developments, but could also pose risks due to catastrophic accidents or misuse. The people working on this problem aim to maximise the chance of a positive outcome, while reducing the chance of catastrophe.

Work on the risks posed by superintelligent machines seems mostly neglected, with total funding for this research well under $10 million a year.

The main opportunity to deal with the problem is to conduct research in philosophy, computer science and mathematics aimed at keeping an AI’s actions and goals in alignment with human intentions, even if it were much more intelligent than us.

In the profile we cover:

The main reasons for and against thinking that the future risks posed by artificial intelligence are a highly pressing problem to work on.

How to use your career to reduce the risks posed by artificial intelligence.

Smoking takes an enormous toll on human health – accounting for about 6% of all ill-health globally according to the best estimates. This is more than HIV and malaria combined. Smoking continues to rise in many developing countries as people become richer and can afford to buy cigarettes.

There are ways to lower smoking rates that have been shown to work elsewhere, such as informing people who are unaware about how much smoking damages their health, as well as simply increasing the price of cigarettes through taxes. These are little used in developing countries, suggesting there is a major opportunity to improve human health by applying the World Health Organization’s recommended anti-tobacco programs.

In the profile we cover:

The main reasons for and against thinking that smoking in the developing world is a highly pressing problem to work on.

Governments, charities, intergovernmental organisations, and social enterprises spend large amounts of money to improve the world but there is currently little research to guide them on what priorities they should focus on at the highest level.

Global priorities research seeks to use new methods to determine in which causes funding to improve the world can have the biggest impact, and make a convincing case about this to people in a position to redirect large amounts of money.

In the profile we cover:

The main reasons for and against thinking that global priorities research is a highly pressing topic to work on.

Every year around ten million people in poorer countries die of illnesses that can be very cheaply prevented or managed, including malaria, HIV, tuberculosis and diarrhoea.

In many cases these diseases or their impacts can be largely eliminated with cheap technologies that are known to work and have existed for decades. Over the last 60 years, death rates from several of these diseases have been more than halved, suggesting particularly clear ways to make progress.

Natural pandemics and new scientifically engineered pathogens could potentially kill millions or even billions of people. Moreover, future progress in synthetic biology is likely to increase the risk and severity of pandemics from engineered pathogens.

But there are promising paths to reducing these risks through regulating potentially dangerous research, improving early detection systems and developing better international emergency response plans.

In the profile we cover:

The main reasons for and against thinking that biosecurity is a highly pressing problem.

50,000,000,000 animals are raised and slaughtered in factory farms globally each year. Most experience extreme levels of suffering over the course of their lives. But there are promising paths to improving the conditions of factory farmed animals and to reducing meat consumption.

In the profile we cover:

The main reasons for and against thinking that factory farming is a highly pressing problem.

Tara saved lives working as a pharmacist in Bhutan – no really we checked, and she totally did – but she nevertheless left to try to find something better.

This is part of our series of profiles of people who changed their career in a major way in order to have more impact because of their exposure to 80,000 Hours.

Today Tara Mac Aulay is the head of operations in the Centre for Effective Altruism. But just two years ago she was working as a pharmacist. How and why did she make this transition? Her career path is sufficiently fascinating it’s worth telling the story form the start.

Tara was extremely conscientious and hard-working from a very young age. As a result she was able to finish high school and start studying at university at the young age of 16, rather than the usual 18 or 19. She managed to do this while at the same time i) redesigning the staff and inventory management for an Australian restaurant chain, then, because this saved them so much money, being promoted to a more senior role to ii) travel around the country to make major changes to failing stores to save them from closure. As a teenager! Needless to say, this entrepreneurialism and ambition allowed her to develop a wide range of professional skills at a young age.

I recently interviewed Dillon Bowen, who runs the EA student group at Tufts University, about how his career plans changed as a result of interacting with 80,000 Hours. Dillon’s original plan was to do a Philosophy PhD and then go into philosophy academia. After going to a talk at Tufts by our co-founder Will MacAskill and receiving career coaching from 80,000 Hours, he started taking classes in economics, now intends to do an Economics PhD instead.

And that’s really worrying. This confidence interval suggests the author puts significant probability on human-level artificial intelligence (HLAI) occurring within 20 years. A survey of the top 100 most cited AI scientists also gave a 10% chance that HLAI is created within ten years (this was the median estimate; the mean was a 10% probability in the next 20 years).

This is like being told there’s a 10% chance aliens will arrive on the earth within the next 20 years.

Making sure this transition goes well could be the most important priority for the human race in the next century. (To read more, see Nick Bostrom’s book, Superintelligence, and this popular introduction by Wait But Why).

We issued a note about AI risk just over a year ago when Bostrom’s book was released. Since then, the field has heated up dramatically.

In January 2014, Google bought Deepmind for $400m. This triggered a wave of investment into companies focused on building human-level AI. A new AI company seems to arrive every week.

Receive our career guide to your inbox, as well as monthly updates on our latest research, events near you and career opportunities.

We're affiliated with the University of Oxford's Future of Humanity Institute and the Oxford Uehiro Centre for Practical Ethics.

We're part of the Centre for Effective Altruism, and work closely with Giving What We Can.

80,000 Hours is part of the Centre for Effective Altruism, a registered charity in England and Wales (Charity Number 1149828) and a registered 501(c)(3) Exempt Organization in the USA (EIN 47-1988398).

We do our best to provide useful information, but how you use the information is up to you. We don’t take responsibility for any loss that results from the use of information on the site. Please consult our full legal disclaimer and privacy policy.

80,000 Hours is part of the Centre for Effective Altruism, a registered charity in England and Wales (Charity Number 1149828) and a registered 501(c)(3) Exempt Organization in the USA (EIN 47-1988398).

We do our best to provide useful information, but how you use the information is up to you. We don’t take responsibility for any loss that results from the use of information on the site. Please consult our full legal disclaimer and privacy policy.