This post discusses cause prioritization from the perspective of downside-focused value systems, i.e. views whose primary concern is the reduction of bads such as suffering. According to such value systems, interventions which reduce risks of astronomical suffering are likely more promising than interventions which primarily reduce extinction risks.

Exponential growth is a common pattern found throughout nature. Yet it is also a pattern that tends not to last, as growth rates tend to decline sooner or later. In biology, this pattern of exponential growth that wanes off is found in everything from the development of individual bodies — for instance, in the growth of […]

We can expect smarter-than-human artificial intelligence (AI) to be better than humans at self-preservation and goal preservation. If we want our actions to have an influence on the very long-term future, we should consider focusing on outcomes with AI.

Setting up the goal systems of advanced AIs in a way that results in benevolent behavior is expected to be difficult. We should account for the possibility that the goal systems of AIs fail to implement our values as originally intended. In this paper, we propose the idea of backup utility functions: Secondary utility functions that are used in case the primary ones “fail”.

Two crucial questions in discussions about the risks of artificial superintelligence are: 1) How much more powerful could an AI become relative to humans, and 2) how easily could superhuman capability be acquired? To answer these questions, this article reviews the literature on human expertise and intelligence and discusses its relevance for AI.

Will we go extinct, or will we succeed in building a flourishing utopia? Discussions about the future trajectory of humanity often center around these two possibilities, which tends to ignore that survival does not always imply utopian outcomes, or that outcomes where humans go extinct could differ tremendously in how much suffering they contain.

The simulation argument suggests a non-trivial chance that most of the copies of ourselves are instantiated in relatively short-lived ancestor simulations run by superintelligent civilizations. If so, when we act to help others in the short run, our good deeds are duplicated many times over. This reasoning dramatically upshifts the relative importance of short-term helping over focusing on the far future.

FRI’s research seeks to identify the best intervention(s) for suffering reducers to work on. Rather than continuing our research indefinitely, we will eventually have to focus our efforts on an intervention directly targeted at improving the world. This report outlines plausible candidates for FRI’s “path to impact” and distills some advice on how current movement building efforts can best prepare for them.

AI outcomes where something goes wrong may differ enormously in the amounts of suffering they contain. An approach that tries to avert the worst of those outcomes seems especially promising because it is currently more neglected than classical AI safety efforts which shoot for a highly specific, “best-case” outcome.

Evaluating the effectiveness of our actions, or even just whether they're beneficial or harmful, is very difficult. One way to deal with uncertainty is to focus on actions that likely have positive effects across many scenarios. This approach often amounts to meta-level activities like promoting positive-sum institutions, reflectiveness, and effective altruism in general.

Some people hold more hopeful beliefs about the world than justified. These include the feeling that life for wild animals isn't so bad and the expectation that humanity's future will reduce more suffering than it creates. By feeding these dreams, optimistic visions of suffering reduction may in fact cause net harm. We should explore ways of increasing empathy that also expose the true extent of suffering in the world.

Artificial intelligence (AI) will likely transform the world later this century. Whether uncontrolled or controlled AIs would create more suffering in expectation is a question to explore further. Regardless, the field of AI safety and policy seems to be a very important space where altruists can make a positive-sum impact along many dimensions.

Space colonization would likely increase rather than decrease total suffering. Because many people care nonetheless about humanity’s spread into the cosmos, we should reduce risks of astronomical future suffering without opposing others’ spacefaring dreams. In general, we recommend to focus on making sure that an intergalactic future will be good if it happens rather than making sure there will be such a future.