Monthly Archives: May 2014

Existential risk is the risk of an event that would wipe out humanity. That means, not just present lives, but all the future ones. Populations can recover from lots of things, but not from complete extinction. If you value future people at all, you might care a lot about even a small reduction in the probability of an extinction event for humanity.

There are two big problems with reasoning about existential risk:

1) By definition, we have never observed the extinction of humanity.

This means that we don’t have an uncensored dataset to do statistics on – extrapolating from the past will give our estimates anthropicbias: people only observe the events that don’t wipe out humanity, but events that wipe out humanity are possible. Therefore, our past observations are a biased subset that make the universe look safer than it is.

2) Our intuitions are terrible about this sort of thing.

To reason about existential risk, we have to tell stories about the future, think about probabilities, sometimes very small ones, and think about very large numbers of people. Our brains are terrible at all these things. For some of these beliefs, there’s just no way to build a “feedback loop” to test your beliefs quickly in small batches – and that’s the main way we know how to figure out when we’re making a mistake.

Moreover, we have to do this on a topic that evokes strong emotions. We’re not talking about the extinction of some random beetle here. We’re talking about you, and me, and everyone we know, and their grandchildren. We’re talking about scary things that we desperately don’t want to believe in, and promising technologies we want to believe are safe. We’re talking about things that sound a lot like religious eschatology. We’re talking about something weird that the world as a whole hasn’t quite yet decided is a normal thing to be worried about.

Can you see how rationality training might be helpful here?

I’m writing this on a flight to Oakland, on my way to CFAR‘s workshop to test out in-development epistemic rationality material. A few months ago, I expressed my excitement at what they’ve already done in the field of instrumental rationality, and my disappointment at the lack of progress on training to help people have accurate beliefs.

During conversations with CFAR staff on this topic, it became clear to me that I cared about this primarily because of Existential Risk, and I strongly encouraged them to develop more epistemic rationality training, because while it’s very hard to do, it seems like the most important thing.

In the meantime, I’ve been trying to figure out what the best way is to train myself to make judgments about existential risks, and about the best thing I can do to help mitigate them.

It turns out that it’s really hard to fix something without a very specific idea of how it’s broken. So I’m going to use the inadequate tools I have, on this flight, without access to experts or the ability to look up data, to build the best bad estimate I can of:

Some sensible people have promotedInternational Tell Your Crush Day. Basically, the idea is to have a day where everyone reveals their secret crushes to the people they have crushes on. Here’s a description from the source itself:

ITCYD was hatched out of a simple and honest desire to see more people be open and honest about the world around them. Specifically, the people part of that world. Everyone, even curmudgeons who will claim otherwise, get crushes on people. And, yes, everyone gets crushed on at different times. Wouldn’t it be rad to be able to tell people you had sparklies for that you had sparklies for them? Wouldn’t it be super-amazing if people would do the same for you?

….for those of you keeping score at home, the correct answer is “YES!”

Some of you, who don’t know me well yet, will have a wrong idea of why this makes me feel incredibly sad, anxious, and scared. Here are some things I am notworried about:

That I will be the object of an unwanted or inconvenient crush.

That someone I have a crush on will not reciprocate my feelings.

That I would face some sort of adverse social consequences for telling someone I have a crush on them

That I’m not allowed to play this game because I have a girlfriend

That I am allowed to play this game (for many reasons including the ones the ITYCD website lists) but people think I’m not

That I don’t know how to tell people how I feel about them

None of these things is distressing to me. Here’s why ITYCD makes me want to go into a dark room and curl up into a ball and hide:

I Don’t Know What a Crush Is

I know what subjective experiences correspond to some emotion-words. Happiness. Sadness. Anger. Fear. But there’s nothing that seems to map neatly onto the feeling of “crush.” (I also have this problem with this description of romantic love.)

Here are some days I’d know how to observe – and would know how to manage the risks of:

Tell Someone You Enjoy Their Conversation or Company Day

Tell Someone You Admire One Of Their Character Traits Day

Tell Someone They Are Nice to Look At Day

Tell Someone They Are So Beautiful It Literally Hurts To Look At Pictures Of Them Day

Tell Someone You Feel Comfortable And Safe Around Them Day

Tell Someone They Turn You On Day

Tell Someone You Feel Like They Understand You Day

Tell Someone If They Ever Truly Needed Your Help You Would Want To Drop Everything To Take Care Of Them Day

I ran these by a friend and she confirmed that for each of these she immediately had a picture in her mind of who she’d be able to truthfully tell it to. But I’m not sure what combination of these add up to “crush” in the relevant sense. ITYCD defines a crush in this way:

[F]or the purposes of ITYCD, a crush is simply a spontaneous (or premeditated, we’re not picky) acknowledgement of the beauty all around us.

I have asked a few people what they mean when they say they have a crush on someone. Every single one of them has a definition that is much, much more specific than this – and they wildly disagree. If I think someone is nice to look at and you admire one of their character traits, but they don’t turn you on and you don’t feel like they understand you, do you have a “crush” on them for the purposes of ITYCD? Do you have to tell them?

ITYCD’s advice to people who don’t recognize any of their feelings as “crushes” is:

Sometimes it’s hard to spot them, if you haven’t been thinking about them for a while. Don’t force a crush just to be able to tell someone you have a crush on them, but we bet if you think about it a bit more, you’ll remember that cute barista who works downtown, or that bike mechanic who fixed up your ride for you.

First of all, it is not obviously okay to make sexual advances to people who are paid to serve you. And I do get the strong sense – as much as the ITYCD website dances around it with words like “sparklies” – that “crush” is supposed to involve sexual attraction.

ITYCD generously implies that if I’m reasonable and a capable adult, I couldn’t possibly need any guidance on this, I’d already know what to do:

Q: Are there people I shouldn’t tell?

A: Unfortunately, probably. The basic rule is: be reasonable! Don’t tell someone you’ve got a crush on them if it means risking your job/marriage/Packers tickets/membership in the Rick Astley fan club… unless of course that’s what you want to do. ISAC recommends caution when telling, among others, your boss, your employees, your professors, your students, or anyone who you’ve got some level of administrative power over or subordination to. But since you’re a capable adult, you get to make the call.

Wow. Such guidance. Very specific. Don’t do it… unless you want to. Isn’t that what we do on literally every other day of the year? I thought this day was supposed to be different.

Second of all, if I guess, then I may be operating under a different definition of the word “crush” than the other person. That’s how this kind of you-know-it-when-you-see-it-ism leads to the next two problems.

False Positives

If I’m just guessing whether my feelings about someone count as a “crush,” and I incorrectly guess that I do, then I may be accidentally saying I have much, much stronger (or just different) feelings than I do. At best I’d be leading them on in some way, which is wrong – but also, if their definition of “crush” is narrower than mine, my calibration for when it’s appropriate to say something will be off and I could make someone seriously uncomfortable or even scared.

False Negatives

If I’m expected to tell all the people I have crushes on, then if someone’s definition of “crush” is wider than mine, I may accidentally give them misleading evidence that I don’t like them, and make them feel bad.

Do You Have To Remind Me That I Am A Defective Human Being?

OK, so there’s this word, and everybody (except me) knows what it means. You won’t tell me what it is, I’ve just got to pretend. If I have normal feelings, then I have obviously experienced what you’re talking about, and anyone else who reports not to is a lying curmudgeon in denial. And I have to use it to figure out whether to have an emotionally loaded conversation with strong feelings. And now that there’s a special day for it, not having that conversation on this day will provide the people who know about it, many of whom I care deeply about in many ways, with strong subjective evidence that I don’t care about or like them.

That’s why I’m not celebrating International Tell Your Crush Day. Please don’t assume I don’t care about you, like you, or whatever it is you think “crush” means, just because I haven’t told you today.

Some People Are Doing It Right

That said, I don’t actually think ITYCD is necessarily terrible for all people. If you need a Schelling point to tell people nice things, go ahead and take it. But I wish that it didn’t have to involve a loaded term like “crush,” and the chipper “this is nice and easy and obvious and nothing will go wrong – unless you’re stupid” tone.

I have at least one friend who is using this as a more generalized opportunity to tell a bunch of people specific nice things. She’s not letting the word “crush” speak for her – she’s specifying which exact feelings she means to express. She also wrote me a note saying that I’m her favorite person she’s not crushing on. And that made my day.

There’s a charity thing going on today where whoever gets the most unique donors of $10 or more by midnight tonight wins $250,000 a bundle of useful services supposedly valued at $250,000.

MIRI, the Machine Intelligence Research Institute, is a group that works on ensuring a positive singularity and friendly AI. They were part-founded by Eliezer Yudkowsky, they’re closely affiliated with Less Wrong, and they briefly employed me). They are currently thirteen only eight donors away from the top place, with only four hours left to go (I think they’re on California time).

EDIT2: Or if you’re interested in eye care for poor Indian children, you can donate to the current leader, Sankara Eye Foundation. Everyone else seems too far behind to catch up

I am not totally sure that MIRI is always the most effective use of your charitable dollars, but I am pretty sure that under these advantageous circumstances, that first $10 has a darned high expected value.

[Offer retracted by MIRI’s request. If you already gave because of what I said, email me and we’ll work something out.]