Rethink Charity

Powering High-Impact Charitable Projects

This is the eighth article in the EA Survey 2017 Series.You can find supporting documents at the bottom of this post, including previous EA surveys conducted by Rethink Charity, and an up-to-date list of articles in the series.

By Anna Mulcahy, Tee Barnett, and Peter Hurford

Summary

We asked self-identified EAs how they first heard about the movement and what resource or tool persuaded them to get more involved. It’s important to bear in mind, however, the limitations related to both the questions and also the respondents ability to recall from possibly long periods ago[1].

The number of people joining the EA movement each year continues to increase year-on-year.

The top five sources of introduction to EA in descending order are ‘Personal Contact’, ‘Lesswrong’, ‘Other book, article, or blog post’, ‘SlateStarCodex’, and ‘80,000 Hours’

As of 2016, LessWrong dropped out of the top five list of introductory sources after being one of the top three from 2009 to 2015.

The top five sources of engagement for new EAs in 2017 in descending order are ‘GiveWell’, ‘Book or Blog’, ‘80,000 Hours’, ‘Personal Contact’, and ‘Giving What we Can’.

What year did EAs first get involved with EA?

EA survey results from 2017 show an increase in the number of new members, confirming trends published in “Is EA Growing? Some EA Growth Metrics for 2017”. Results show growth of nearly 20% in the number of new recruits to the community for 2016, compared to 2015. This certainly reflects gains in recruitment year-on-year, though without efforts to track attrition rates it is possible that the total community growth could be less than what is suggested here.

How did people first hear about EA?

All-time figures for first introductions to EA were topped by ‘Personal Contact’ and “Lesswrong’ with ‘Other book, article, or blog post’ coming in a distant third. Scott Alexander’s SlateStarCodex (SSC) and 80,000 Hours round out the top five. Important to note is a considerable proportion of individuals who selected ‘Other’, which would place it as the third most popular answer if counted among specific referral sources.

Responses were then cross-referenced against the question, “In roughly which year did you first get involved in EA?”. This allowed for the 2017 results to be interpreted within a longer arc of EA surveys conducted in the last few years, and provided some indication about how the most successful sources for spreading the word about EA have changed over time.

We can also see how referrers have changed over time by cross-referencing people’s self-report of how they got involved with the year they report joining the movement. When interpreting the 2017 results within this context, we find that getting introduced to EA through personal networks has historically been most common (Table 3).

As for year-on-year trends according to particular referral sources, we can find several examples of noteworthy changes over time. For instance, Lesswrong was a wellspring of new EAs for several years before the community faded. 80,000 Hours is typically among the top referrers, and while SlateStarCodex has as also been important over the years according to this survey, potential for survey bias due to over-sampling SSC readers persists.

From 2009 to 2011, Giving What We Can ranked highly in response to the question “How did you first hear about EA?”. However, after 2011 it progressively fell in popularity and did not even rank in the top five ways of first hearing about EA from 2014 to 2016. In addition, the number of people who learned about EA through 80,000 hours almost doubled from 2014 to 2015 (see Table 4)[2]. Slate Star Codex has also shown increasing success as a referral source for EA since 2014 (see Table 5). Again, care must be used when interpreting these trends, as there have been fluctuations in how much each group promoted the EA Survey.

As mentioned previously, despite LessWrong dropping out of the top five in 2016, the historical strength of the rationalist website in drawing EA-adjacent individuals suggests that it may have been an obvious choice for EA Grants to support the newest iteration of LessWrong.

Comparison with a Survey of the EA Facebook Group

The EA Facebook group has become a popular place for the EA Community. Indeed, 54.6% of EAs in our survey sample report being in the group, and almost 18% of EA survey respondents were referred from Facebook. Notably, when people join the EA Facebook group, as a condition of joining, every member is asked to report how they heard about EA as freeform text. Julia Wise and other EA FB moderators collected a convenience sample of 100 responses collected in late 2017 and produced the following results:

To compare this to our data, we selected the 406 EAs who self-reported being a member of the EA Facebook group and who said they joined in 2016 or 2017 (though this would only go up to April-June 2017 when the survey was active). Among this subsample in our survey, the top five results were 19% saying personal contact, 17% saying “other”, 10% saying 80,000 Hours, 7% saying Doing Good Better, and 6% saying a TED Talk. This matches closely with the results gathered from Facebook despite a different data collection method (forced response for group membership vs. voluntary survey taking) and reporting methods (self-report from choices including others vs. self-reported freeform text with no prompts).

What got people more involved with EA?

Respondents were also asked what motivated them to get involved with EA. While the previous question can indicate the reach and accessibility of EA resources, this question can be used to indicated how effective these resources are at persuading people to join the EA community and actively participate.

Introduction sources and sources of further engagement are not always one in the same. As seen in Table 7, personal networking did not come out as the top source for actually getting people involved in EA, though it remains within the top five. Once introduced to EA, it would appear GiveWell, books and/or blogs, and 80,000 Hours are the three most potent ways to keep engage new EAs. This may come as no surprise considering these answer options offer a wealth of in-depth information. LessWrong would presumably also fall into this camp, but the rationalist website may have fallen down the list due to reasons cited above. EA Global (EAG) performed quite well considering the relatively brief amount of time new EAs have to engage at a given conference.

Endnotes

[1] As mentioned in previous articles, care should be taken when interpreting EA survey results. Questions to identify where people first heard about EA are open to significant human error as respondents are required to rely on memory and recall something that may have happened up to 5 or more years ago. Furthermore, respondents could have heard about EA from multiple sources in a short period of time, but may not be able to pinpoint exactly which of those sources they heard about it from first. Having ‘cannot remember’ as an option can only reduce errors from memory recall up to a point.

The same potential for error applies when asking respondents to recall what caused them to actually get involved in EA. Although for this question they were given the opportunity to select multiple answers, as multiple factors often contribute to such a decision, so it relied less on accurate recall of a single, specific event.

[2] This may be the case for a few reasons. 80,000 Hours assisted this year in distributing the survey, which was not the case in 2016 because no EA survey was conducted. According to CEO and Co-founder, Ben Todd, 80,000 Hours web traffic nearly doubled each year for the past few years. And finally, the Effective Altruism Facebook group survey posted by Julia Wise illustrates the popularity of 80,000 Hours as a popular referral source among new members.

[3]: The full text of the question was “Which factors were important in ‘getting you into’ Effective Altruism, or altering your actions in its direction? Check all that apply.”

Credits

Post written by Anna Mulcahy, Tee Barnett, and Peter Hurford, with edits from Ben Todd.

The annual EA Survey is a volunteer-led project of Rethink Charity that has become a benchmark for better understanding the EA community. A special thanks to Ellen McGeoch, Peter Hurford, and Tom Ash for leading and coordinating the 2017 EA Survey. Additional acknowledgements include: Michael Sadowsky and Gina Stuessy for their contribution to the construction and distribution of the survey, Peter Hurford and Michael Sadowsky for conducting the data analysis, and our volunteers who assisted with beta testing and reporting: Heather Adams, Mario Beraha, Jackie Burhans, and Nick Yeretsian.

Thanks once again to Ellen McGeoch for her presentation of the 2017 EA Survey results at EA Global San Francisco.

We would also like to express our appreciation to the Centre for Effective Altruism, Scott Alexander via SlateStarCodex, 80,000 Hours, EA London, and Animal Charity Evaluators for their assistance in distributing the survey. Thanks also to everyone who took and shared the survey.

This is the seventh article in the EA Survey 2017 Series.You can find supporting documents at the bottom of this post, including our prior EA surveys, and an up-to-date list of articles in the series.

Summary

We use past survey data to shed light on community shifts in cause area preferences over time.

Our evidence suggests that EAs are becoming more favorable toward AI and less favorable toward politics.

EAs in both the 2015 and 2017 surveys shifted away from viewing poverty as a “top” or “near top” cause.

Newcomers in the 2015 survey were less accepting of global poverty than veterans. However, the reverse was true in the 2017 survey, with newcomers being more accepting of global poverty than veterans.

There is no indication that EAs are getting less interested in animal welfare with time.

Cause Preference Shifts

Our previous posts in this series were largely descriptive, often reporting on 2015 and 2016 to provide an approximate snapshot of the current EA community. As the series progresses into late 2017, we’ll look to extract further insight from the data, which will include various longitudinal analyses, commentary on the Pledge, and potentially other angles upon request.

We turn first to a commonly held narrative within the community – that new EAs are typically attracted to poverty relief as a top cause initially, but subsequently branch out after exploring other EA cause areas. An extension of this line of thinking credits increased familiarity with EA for making AI more palatable as a cause area. In other words, the top of the EA outreach funnel is most relatable to newcomers (poverty), while cause areas toward the bottom of the funnel (AI) seem more appealing with time and further exposure. (For example, see Michael Plant’s post “The marketing gap and a plea for moral inclusivity”.) While we previously reported higher support for global poverty as a top cause, we find reason to support some version of a narrative suggesting that EAs are shifting away from global poverty.

There are two ways we’ve looked at changes in preference toward causes over time. First, we took the information on what year EAs joined the community, and compared the cause preferences of earlier EAs to newcomers. Our second method involved taking the population of EAs who took the EA Survey in both 2015 and 2017 and seeing how the same people changed their opinions of their top cause over this two year gap. The first method has a larger sample size, while the second version captures intrapersonal attitude shifts over time. Both tell a similar tale.

Using the longitudinal method, there were 184 people who took both the 2015 and 2017 EA Surveys that we could match (using a hashed email address to preserve anonymity). To get a quick overview of cause preference change over time, we looked at the number of people who shifted toward a cause (they previously had not considered the cause to be a “top priority” or “near the top priority” in 2015, but now do as of 2017) and subtracted the number of people who shifted away from a cause (they previously had considered the cause to be “top” or “near top” and now don’t). This gave us a number we called a “net shift” from a cause.

Cause area preferences fluctuated slightly between the 2015 and 2017 EA surveys (Table 1). Poverty remains the clear community favorite, although the net shift in preference broken down by cause area reveals that interest has been waning in poverty since the 2015 EA survey, with a net shift of -8. Interestingly, politics has hemorrhaged the most interest (-13) in the wake of Brexit, Trump’s victory, and other significant political developments in traditional EA hubs. The biggest winner in net gains is AI (+29) and non-AI far future (+14), which suggests at least some directional movement toward long-term concerns over time.

We were compelled to take a closer look at the dropping interest in poverty, particularly due to its continued popularity in the aggregate and traditional status as an EA mainstay. Between the 2015 and the 2017 surveys, 14.13% of EAs in the longitudinal sample changed their mind about how much importance should be placed on the cause (Table 2), with 9.24% of these EAs no longer considering poverty as a “top” or “near top” cause, and 4.89% of EAs upgrading their estimation of poverty’s importance.

However, there has been more movement within the distinction between “top” and “near top”, with 19.02% of EAs in the longitudinal sample relegating poverty from being the top cause two years later and only 5.98% of EAs upgrading their estimation of poverty as the most important cause area (Table 3).

To look at this from another perspective, we took the 2017 EA Survey population and distinguished between whether an EA was more of a “veteran” who learned about EA in 2013 or earlier or was more of a relative newcomer who learned about EA in 2014 or later[1]. The hypothesis is that veteran EAs would have had more time to shift their beliefs in causes and may be predictive of how newcomers will eventually shift.

Taking initial preferences into consideration, EAs who joined in 2013 or earlier were far less likely to rank poverty as the “top” or “near top” priority than EAs who joined in 2014 or later (Table 4), though a majority of these veteran EAs still ranked poverty as the “top” or “near top” cause.

One potential explanation for this shift might not be a genuine change in opinion over time, but instead that veteran EAs were always less likely to be into poverty, whereas newer EAs are a lot more likely to be into poverty. To check our base assumption about whether there has been a significant influx of poverty-focused EAs in recent years, we looked back at the 2015 EA Survey and compared it to the 2017 EA Survey (Table 5).

As of the 2015 Survey, newcomers were actually relatively less accepting of global poverty than the veterans, but this effect reverses as of the 2017 EA Survey. This could point to a difference in attitudes for newcomers in 2015 and 2017 skewing the data, rather than newcomers from 2015 changing their minds over time.

The data is not entirely clear on whether initially interested EAs change their views away from poverty with time. The perceived separation between veteran EAs being less poverty-focused may be down to initial dispositions, rather than later conversions. The 2017 EA survey data does suggest that most newcomers enter the movement interested in poverty, which may have implications for movement building organizations to bear in mind.

Attitudes Toward AI

Turning to AI, not only has resistance to devoting resources to AI safety reduced substantially since the 2015 EA Survey, but we showed that this set of concerns is now actively competing with other cause areas for top priority billing.

There were more people changing their minds on AI than global poverty (Table 6), with 19.57% of EAs in our longitudinal sample choosing to upgrade the importance of AI in their view to a “top” or “near top” cause and only 3.8% of EAs choosing to downgrade it out of “top” and “near top”. When looking at just top cause area preference, the trends were roughly similar, with 13.04% of EAs in the longitudinal sample promoting AI to the top cause and 7.07% demoting AI from top cause to something else.

Among those veteran EAs who joined in 2013 or earlier, the support for AI as a “top” or “near top” priority was closer to 50-50, whereas for EAs who joined in 2014 or later, there is less support for AI as a “top” or “near top” cause (Table 7). The net shift of aggregate interest toward AI (Table 1), a broad trend favoring AI (Table 6), combined with our knowledge that newer EAs favor AI relatively less (Table 7), would seem to suggest that more exposure to EA increases the likelihood of becoming more inclined to support AI safety over time.

Attitudes Toward Animal Welfare

We were also curious to check the same for animal rights, to see how EA interest in helping animals as a cause has changed over the years.

Here we see that among the 2017 EA Survey respondents, unlike with AI, there is no statistically significant difference between the rate at which newcomers and veterans support animal rights (Table 9). Furthermore, there has been a net shift toward animal welfare among those who took both the 2015 and 2017 EA Surveys (Table 8). Thus, suggestions that EAs are getting less interested in animal welfare over time does not seem to be confirmed by EA Survey data.

Among the 2017 EA Survey respondents, newcomers to EA are relatively more likely to support politics than veterans, though the majority of both newcomers and veterans do not support politics as a “top” or “near top” cause (Table 11). Similarly, among those who took both the 2015 and 2017 EA Surveys, people are shifting away from thinking of politics as a “top” or “near top” cause (Table 10). This may mean that while politics is less popular as an EA cause overall, EAs tend to shift away from it over time. Likewise, it is interesting that it seems like contentious developments of late may have not had any sort of energizing effect on getting EAs interested in politics, as far as we can tell in this survey data.

Endnotes

[1]: This effect is statistically significant at p < 0.00001 for both. We chose 2013 because we felt it properly conveyed “veteran” status before a lot of popular growth in EA in 2014, but this effect remains the same in direction and statistical significance, with similar strength, regardless of your choice for cut-off year (tested with 2011, 2012, 2013, 2014, and 2015 as cut-off years).

Credits

Post written by Peter Hurford and Tee Barnett

The annual EA Survey is a volunteer-led project of Rethink Charity that has become a benchmark for better understanding the EA community. A special thanks to Ellen McGeoch, Peter Hurford, and Tom Ash for leading and coordinating the 2017 EA Survey. Additional acknowledgements include: Michael Sadowsky and Gina Stuessy for their contribution to the construction and distribution of the survey, Peter Hurford and Michael Sadowsky for conducting the data analysis, and our volunteers who assisted with beta testing and reporting: Heather Adams, Mario Beraha, Jackie Burhans, and Nick Yeretsian.

Thanks once again to Ellen McGeoch for her presentation of the 2017 EA Survey results at EA Global San Francisco.

We would also like to express our appreciation to the Centre for Effective Altruism, Scott Alexander via SlateStarCodex, 80,000 Hours, EA London, and Animal Charity Evaluators for their assistance in distributing the survey. Thanks also to everyone who took and shared the survey.

The annual EA Survey is a volunteer-led project of Rethink Charity that has become a benchmark for better understanding the EA community. This is the sixth article in our multi-part EA Survey 2017 Series. You can find supporting documents at the bottom of this post, including prior EA surveys, and an up-to-date list of articles in the EA Survey 2017 Series.

Could you, however loosely, be described as an “Effective Altruist”?

Several respondents support the underlying principles of the EA movement, but many suggested that they did not consider themselves part of the community because of their disagreement with some of the ideas, or their lack of donations to effective charities (often due to financial difficulties or perceived lack of commitment). Various respondents also seemed to view EA as a lofty, principle-based lifestyle that they had not yet attained and were therefore hesitant to label themselves “effective altruists.” A few comments suggested that the term “effective altruist” implied an underlying pretentiousness that respondents were unwilling to associate with.

If there was a local group near your home, would you attend?

For this question, people tended to respond in one of two ways: respondents in the first group tended to be active participants and/or leaders in their local EA group. Those that did not live in an area with a local EA group expressed interest in starting such a community. Respondents in the second group showed interest in attending occasional meetings. At the same time, these respondents also expressed some ambivalence about attending meetings. Distance and scheduling were common concerns; people also wanted to know how effective and structured the group meetings would be in reaching practical outcomes.

How welcoming do you find the EA community?

Responses varied widely based on the region and the particular forum being referenced. People generally commented that the online community feels off-putting to new members as the topics discussed are very specialized and members tend to be very well-informed. As a typical response went: “Sometimes the jargon and in depth conversations can be a bit alienating to someone without a philosophy or economics background.” Relating to this concern, a few respondents commented that it would be best to create a separate, more open space dedicated to bringing new members up to speed on EA ideas.

Another common theme was that the EA community tends to attract members with similar ethnic, socioeconomic, and educational backgrounds. Respondents noted that the lack of diversity often made it difficult for those outside the demographic to feel comfortable in the EA community.

Do insecurities about not being ‘EA enough’ sometimes prevent you from taking action or participating more in the EA community?

Many respondents expressed a certain degree of guilt for not having “done enough” as an effective altruist, especially when compared to more dedicated members of the EA community. This insecurity seems to largely be the result of internal sentiments (e.g. feeling that they do not have anything worthwhile to contribute), and at least partly attributable to a dynamic inside EA groups that does not fully accommodate new members.

Others expressed satisfaction with their current level of giving and the extent to which they had embraced EA ideas in their daily life.

How can we improve the EA survey?

In this question, respondents highlighted four critical areas of improvement for the survey content. First, they were concerned that so many of the questions asked about donations and participants’ income. According to responses, these questions were tedious and reflected poorly on the nature of EA. Second, several respondents raised serious concerns that the multiple choice questions did not account for all possible answers; for instance, one person noted that the careers list did not include a “retail” option but did have a “business” and “manual labor” option, appearing to exclude individuals of lower income classes. These respondents suggested that more multiple choice questions include an option for “other.” Furthermore, responses noted that many of the questions did not distinguish between EA as a set of principles for doing good and the EA community. Finally, respondents consistently noted that the survey was much longer than advertised and actually took 30-45 min.

Respondents also had specific complaints about the formatting of the survey. First, several voiced frustration that the positioning and color coding of the “Exit & Clear survey” caused them to mistake it for the “next” button and accidentally delete their responses. Others noted that it would be very convenient, both for the respondents and the writers of the survey, to sync individuals’ data from the GWWC My Giving website, eliminating the need for all the questions about donations and income. The survey also caused some problems for active participants of the EA movement. For questions that gauged respondents’ interest in setting up an EAHub profile or subscribing to a newsletter, there was no option for those who had already completed these items.

How did you hear about this survey?

The vast majority of respondents heard about the survey via the Slate Star Codex blog and open threads. Respondents frequently recalled accessing the survey via Facebook group pages such as the GWWC Community page, the Effective Animal Advocacy Discussion page, local EA group pages, and the Dank EA Memes page. A significant number heard about the survey directly from EA-affiliated organizations, including 80000 Hours, Rethink Charity (formerly known as Dot Impact), Students for High-Impact Charity, and Giving What We Can; leaders of these organizations either sent out email newsletters with the survey link or directly contacted individuals with information about the survey.

Credits

Post written by June Lee, with edits from Tee Barnett and analysis from Peter Hurford.

A special thanks to Ellen McGeoch, Peter Hurford, and Tom Ash for leading and coordinating the 2017 EA Survey. Additional acknowledgements include: Michael Sadowsky and Gina Stuessy for their contribution to the construction and distribution of the survey, Peter Hurford and Michael Sadowsky for conducting the data analysis, and our volunteers who assisted with beta testing and reporting: Heather Adams, Mario Beraha, Jackie Burhans, and Nick Yeretsian.

Thanks once again to Ellen McGeoch for her presentation of the 2017 EA Survey results at EA Global San Francisco.

We would also like to express our appreciation to the Centre for Effective Altruism, Scott Alexander via SlateStarCodex, 80,000 Hours, EA London, and Animal Charity Evaluators for their assistance in distributing the survey. Thanks also to everyone who took and shared the survey.

The annual EA Survey is a volunteer-led project ofRethink Charity that has become a benchmark for better understanding the EA community. This is the fifth article in our multi-part EA Survey 2017 Series. You can find supporting documents at the bottom of this post, including our previous piece on community demographics, prior EA surveys, and an up-to-date list of articles in the EA Survey 2017 Series.

This article brings EA demographics back by popular demand. As in, demand for the metrics not covered in the previous post. We hope you enjoy this second look.

Race

The survey respondents identified as white by a wide majority. Among the 1,069 who self-identified regarding race, 88.9% identified as white, 0.7% identified as black, 3.3% identified as hispanic, 7.0% identified as asian, and 621 respondents preferred not to answer the question. It was possible to identify with as many races as one wanted, but only 3.59% answered ‘Yes’ to self-identify as more than one race, and only one person (0.09%) identified with three races.

While diversity comes in many forms, especially in a definitional sense, EA is unlikely to be characterized as racially diverse according to this survey. There may be considerable margin for error in these findings, not the least because such a large proportion of respondents did not answer. But the trope of EA being a predominantly white (89%) and male (70.1%) community, however, is not likely to fade anytime soon without directed effort.

A longitudinal analysis of the community’s racial composition cannot be conducted because no data on race was gathered in the 2015 survey.

Want to contribute more to this discussion? We recommend joining the Diversity & Inclusion in EA group on Facebook.

Race and Geographic Location

A crosstab of declared racial identity according to location revealed a vast white majority across the top five EA hubs around the world. New York City emerged as the most racially diverse EA hub in the community. This was statistically significant with p = 0.02, but it’s not clear how much we can read into this.

Politics

Left-leaning EAs composed 64.8% of respondents, while ‘Centre’ (8.1%), ‘Centre Right’ and ‘Right’ (3.3%) accounted for a considerably smaller portion of the sample. Libertarian EAs constitute a sizeable proportion of the sample (8.7%) a small group (6%) explicitly chose not to answer, and 9% refused to identify with any of the political spectrum. These percentages do not include the 785 people who took the survey but did not answer this question.

Data on political preference was collected but not published in the 2015 EA Survey report, allowing us in 2017 to present longitudinal data on community-wide shifts in political orientation.

From 2015 to 2017, the survey indicates a slight shift away from the political left in the EA community. The tables above show 27.27% of the 2015 ‘Left’ moved to the ‘Centre Left’, and 5.88% of the ‘Centre Left’ went “Centre”. There was also some polarization, as 46.15% of the “Centre” moved “Centre Left”.

Want to contribute more to this discussion? We recommend joining the Effective Altruists Discuss Politics group on Facebook.

Politics and cause area preference

When looking at the relationship between politics and other areas, we broke down political orientation into whether someone identified with the “Left” (i.e. they said they were “Left” or “Centre Left”) or did not identify with the left (i.e., they picked a different option like, “Centre”, “Centre Right”, “Right”, “Libertarian”). “Other” and “Prefer not to answer” were dropped from this variable. We found 682 respondents who were associated with a left-leaning position (left), 212 respondents who were not associated with a left-leaning position (non-left), and 943 people with no position.

A crosstab of political orientation and cause area preference revealed that individuals on the left are more likely to be interested in politics (28% of people on the left rate politics as a top or “near top” cause, compared to 22% of people not on the left), poverty (78% of people on the left rate poverty as a top or “near top” cause, compared to 72% of people not on the left), animal welfare (41% of people on the left say animal welfare is top or near top compared to only 28% of the non-left), and environmentalism (42% of people on the left say environmentalism is top or “near top”. compared to 21% of non-left).

Conversely, people on the left are less likely to care about AI (42% of people on the left rate AI as top or “near top” compared to 47% of people not on the left).

Politics and geographic location

Despite the San Francisco Bay Area being anecdotally associated with libertarians, it had the highest amount of people identifying with the left, with 82.9% of Bay Area respondents. Of the other five largest EA cities, London was 80.85% left, Oxford was 76.92% left and Boston was 73.53% left, and New York City was 63.64% left. However, despite these percentages of left appearing quite different, there was no statistically significant trend in left vs. non-left that we could pick up in our data.

Politics and dietary habits

Results show a significant difference according to political affiliation, where 48.9% on the left identified as vegetarian or vegan, while only 29% on the non-left did.

This makes sense in the light of the above, looking at politics and cause area preference, where we see a significantly greater proportion (41%) of people on the left putting a high priority on animal welfare, compared to a smaller proportion sharing that level of priority from those on the non-left (28%).

Age and cause area preference

Using the median age of 27 as a dividing point, those below the median grouped as ‘younger’ and those above the median as ‘older’, we compared cause area preference in these two groups. The group younger than the median age showed a preference for AI (53.1% compared to 37.9% of older people) and less of a preference for poverty (72% vs. 78% of older people).

Employment status

Employment status responses were lead by for-profit work (43.7%) and non-profit organizations (17.0%). There were similar numbers for self-employed (9.5%) and academics work (9.6%). Unemployed respondents made up 7.7%, while 6.8% reported working for a government entity, and 1.2% were homemakers. Those who are financially independent, through savings, passive income or a providing partner accounted for 4.6%.

Field of study

Respondents were allowed to select more than one field of study. Most popular fields among EA’s, by a significant margin, proved to be computer science (18.9%) and maths (16.1%). Following that, philosophy (9.9%), other sciences (9.2%), social sciences (8.6%) and economics (8.4%). Less often chosen were the fields of humanities (7.1%), engineering (6.9%), physics (6.7%) and finally medicine (2.8%).

Year joined EA

Pardoning 2017 for being the current year, the last few years appear to have been strong for EA recruitment, though there may also be a survivorship bias with EAs who joined in previous years no longer identifying with EA or take the EA survey. Post-2013, we see double-digit percentage growth in the number self-identified EAs joining the community.

Credits

Post written by Katie Gertsch and Tee Barnett, with edits and analysis from Peter Hurford.

A special thanks to Ellen McGeoch, Peter Hurford, and Tom Ash for leading and coordinating the 2017 EA Survey. Additional acknowledgements include: Michael Sadowsky and Gina Stuessy for their contribution to the construction and distribution of the survey, Peter Hurford and Michael Sadowsky for conducting the data analysis, and our volunteers who assisted with beta testing and reporting: Heather Adams, Mario Beraha, Jackie Burhans, and Nick Yeretsian.

Thanks once again to Ellen McGeoch for her presentation of the 2017 EA Survey results at EA Global San Francisco.

We would also like to express our appreciation to the Centre for Effective Altruism, Scott Alexander via SlateStarCodex, 80,000 Hours, EA London, and Animal Charity Evaluators for their assistance in distributing the survey. Thanks also to everyone who took and shared the survey.

The annual EA Survey is a volunteer-led project ofRethink Charity that has become a benchmark for better understanding the EA community. This post is the fourth in a multi-part series intended to provide the survey results in a more digestible and engaging format. You can find key supporting documents, including prior EA surveys and an up-to-date list of articles in the EA Survey 2017 Series, at the bottom of this post.

Our earlier post presented declared preferences among respondents, and donation reporting allows us to further contextualize behavioral trends within the EA community. The most recent survey of 1019 individuals collected donation data on both 2015 and 2016 donations. The survey was not distributed in 2016.

This post aims to compare donation data of the EA community, and within a couple specific subpopulations. You can find donation data according to cause area and organization preference in our “Cause Area Preferences” post.

Points of Interest

Self­-described EAs in our survey reported more than $6.6M in total donations to effective charities for 2015, and more than $9.8M in 2016.

Average donation amounts between 2015 and 2016 were heavily skewed upward by major donors, but the median donation amount rose $118.68.

Donors parting with $655.17 or more fall within the top 50% of EA donors. Gifts totalling $12,500 or more are among the top 10%.

405 people who identify their career plan as “Earning to give” (ETG). In 2015, these people accounted for 63.0% of total reported donations. In 2016, ETG donations constituted 57.3% of total reported donations.

How Much are EAs Donating?

Relatively high average donation rates seem to be commonly associated with effective altruists. So how much are EAs donating?

Self­-described EAs in our survey reported more than $6.6m in total donations to effective charities for 2015, and more than $9.8m in 2016. We standardized all the donations into US dollars and found that the average 2015 donation was $6,498 among respondents, while the average donation in 2016 was $9,510. These seemingly impressive are seriously skewed upward by a few major donors.

The more informative metric, the median donation, was $250 in 2015, and $655 in 2016. This increase was probably due, in part, to the fact that the survey was released in 2017, and so respondents were probably more involved with the movement in 2016 than in 2015 on average. We see evidence of this when comparing donation activity between years. The survey reveals that 150 respondents donated in 2016, but not in 2015. Only 29 donated in 2015, but not 2016. A total of 999 people provided data for both 2015 and 2016 donations.

Although personal donation amounts fluctuated between 2015 and 2016, the mean donation amount per person increased by $3,663.68. This obviously includes a huge variance, however, the median donation amount also increased by $118.68[1].

To help visualize the distribution of donation amounts, let’s look at it in terms of deciles. In other words, how much you would have to donate to be in the top X% of donors based on the reports that we have from the 2016 data.

In order to top the highest donation in our registry, you would have to donate over $1,934,550.

According to the survey, EA donations are highly skewed toward a handful of major donors. Many individuals could make it into the top 50% of EA donors by donating a small percentage of their income, but only a distinct minority are capable of making it into the top 1%.

Donations are clearly affected by student status. In 2016, the median donation of non-­students was $1,538, compared to the median donation of students at $154. The 258 students who donated gave $252,339.60 in total, while the 482 non-students who donated gave $7,242,580.64.

These donations may be over­reported, given that who donate less might be less inclined to share that information. We found, however, a relatively more forthcoming sample than expected. Among those who reported on donations, 29% in 2015 and 16.4% in 2016 reported donating $0.

Percentage of Income Donated

The mean percentage of income donated was 7.98% of in 2016[2], but again this is skewed. The median is 4.28%. While this may seem low when benchmarked against the 10% commitment of the Giving What We Can pledge, it is higher than the United States national average of around 2% of GDP[3]. To better illustrate the point, let’s look at how many people donate at or above a certain amount of income. Since many neglected to reveal their income, or made less than $10,000, this is based on a sample of 597 EAs.

It is also possible that people compensate for 2016 donation deficits by donating more at different times. Note also that this finding also doesn’t capture the EAs that are saving now while waiting for better causes to donate to later.

Donations Among Earning to Give

Perhaps one of the more prescient questions in the community is how much ETG individuals are donating. This question includes all individuals who plan to pursue, or are already involved in ETG careers. In 2015, donations among the 405 ETG individuals in our survey totaled $4,210,633.29. In 2016, donations totaled $5,672,334.74.

The median donation amount in 2015 for 255 ETGnon-students is $237.65. For 2016, the median amount is $798.57, which is actually less than the median donation for non-students generally. This suggests that many ETG individuals are aiming to give later, and perhaps building career capital in the meantime.

We can break this down further by analyzing how EAs responded to “Do you believe that – for you at the moment – it is better to act now or invest to act better later?”. Among the 148ETG non-students who answered “Act now”, the median donation was $4,510. Among the 51non-students who answered “Act later”, the median donation was $712.08. This suggests that the low median donation for earning to give is due to people investing to give later.

Longitudinal Analysis

To look at how donation behavior changes between a subset of individuals, rather than among EA as a whole, we were able to follow a specific group of EAs who took both the 2015 and 2017 EA Surveys[4].

The table above reflects consistent year-on-year growth in donations among 184 individuals we tracked across the last three EA surveys. It’s worth noting, however, there is survivorship bias in this group, as EAs who cease donating might also be less likely to take the 2017 EA Survey.

Endnotes

[1]: The median increase is smaller than the difference between the medians for each year, because it only includes people who donated in both years.

[2]: Percent income percentages were performed only for people with income greater than $10K, as donations as a percentage of income became quite absurd with low incomes, including many people donating without any income at all. This was chosen prior to any analysis. Income here refers to self-reported individual income, as opposed to household income.

[4]: The 2014 and 2015 EA surveys covered donation data of the prior year, while the 2017 EA survey covered 2015 and 2016 donation data. For everyone in the 2015 EA Survey and 2017 EA Survey who provided an email address, we hashed their email address using the MD5 hashing function and matched up email addresses between survey data while still ensuring anonymity. This variable is available as `ea_id` in all the public datasets. 180 people could be matched up between 2015 and 2017 surveys and 18 people could be matched up between all three surveys (2014, 2015, and 2017).

Credits

Post written by Huw Thomas, with edits from Tee Barnett and analysis from Peter Hurford.

A special thanks to Ellen McGeoch, Peter Hurford, and Tom Ash for leading and coordinating the 2017 EA Survey. Additional acknowledgements include: Michael Sadowsky and Gina Stuessy for their contribution to the construction and distribution of the survey, Peter Hurford and Michael Sadowsky for conducting the data analysis, and our volunteers who assisted with beta testing and reporting: Heather Adams, Mario Beraha, Jackie Burhans, and Nick Yeretsian.

Thanks once again to Ellen McGeoch for her presentation of the 2017 EA Survey results at EA Global San Francisco.

We would also like to express our appreciation to the Centre for Effective Altruism, Scott Alexander via SlateStarCodex, 80,000 Hours, EA London, and Animal Charity Evaluators for their assistance in distributing the survey. Thanks also to everyone who took and shared the survey.

The annual EA Survey is a volunteer-led project of Rethink Charity that has become a benchmark for better understanding the EA community. This post is the third in a multi-part series intended to provide the survey results in a more digestible and engaging format. You can find key supporting documents, including prior EA surveys and an up-to-date list of articles in the EA Survey 2017 Series, at the bottom of this post.

Significant plurality within the community means EAs have different ideas as to which causes will have the most impact. As in previous years, we asked which causes people think are important, first presenting a series of causes, and then letting people answer whether they feel the cause is “The top priority”, “Near the top priority”, through to “I do not think any EA resources should be devoted to this cause”.

As in previous years (2014 and 2015), poverty was overwhelmingly identified as the top priority by respondents. As can be seen in the chart above, 601 EAs (or nearly 41%) identified poverty as the top priority, followed by cause prioritization (~19%) and AI (~16%). Poverty was also the most common choice of near-top priority (~14%), followed closely by cause prioritization (~13%) and non-AI far future existential risk (~12%).

Causes that many EAs thought no resources should go toward included politics, animal welfare, environmentalism, and AI. There were very few people who did not want to put any EA resources into cause prioritization, poverty, and meta causes.

Overall, cause prioritisation among EAs reflects very similar trends to the results from 2014 and 2015. However, the proportion of EAs who thought that no resources should go towards AI has dropped significantly since the 2014 and 2015 survey, down from ~16% to ~6%. We find this supports the common assumption that EA has become increasingly accepting of AI as an important cause area to support. Global poverty continues to be overwhelmingly identified as top-priority despite this noticeable softening toward AI.

How are Cause Area Priorities Correlated with Demographics?

The degree to which individuals prioritised the far future varied considerably according to gender identity. Only 1.6% of donating women said that they donated to far future, compared to 10.9% of men (p = 0.00015). Donations to organisations focusing on poverty were less varied according to gender, with 46% of women donating to poverty, compared to 50.6% of men (not statistically significant).

The identification of animal welfare as the top priority was highly correlated with the amount of meat that EAs were eating. The chart below shows the proportion of EAs who identified animal welfare as a top priority according to gender. Considerably more EAs who identified as female ranked animal welfare as a top or near top priority (~47%), as opposed to ~35% males. The second chart shows the dietary choices of those who identified animal welfare as the top priority. Those who identified animal welfare as top or near top priority were overwhelmingly vegetarian or vegan (~57%), much more than the EA rate of ~20%, which looks promising when compared to the estimated proportion of US citizens aged 17+ who are vegetarian or vegan (2%).

The survey also indicated a clustering of cause prioritisation according to geography. Most notably, 62.7% of respondents in the San Francisco Bay area thought that AI was a top or near top priority, compared to 44.6% of respondents outside the Bay (p = 0.01). In all other locations in which more than 10 EAs reported living, cause prioritisation or poverty (and more often the latter) were the two most popular cause areas. For years, the San Francisco Bay area has been known anecdotally as a hotbed of interest in artificial intelligence. Interesting to note would be the concentration of EA-aligned organizations located in an area that heavily favors AI as a cause area [1].

Furthermore, environmentalism was one of the lowest ranking cause areas in the Bay Area, New York, Seattle and Berlin. However, it was more favored elsewhere, including in Oxford and Cambridge (UK), where it was ranked second highest. Also, with the exception of Cambridge (UK) and New York, politics was consistently ranked either lowest or second lowest.

[1] This paragraph was revised on September 9, 2017 to reflect the Bay Area as an outlier in terms of the amount of support for AI, rather than declaring AI an outlier as a cause area.

Donations by Cause Area

Donation reporting provides valuable data on behavioral trends within EA. In this instance, we were interested to see what tangible efforts EAs were making toward supporting specific cause areas. We presented a list and asked to which organization EAs donated. We will write a post about general donation habits of EAs in the next survey.

As in 2014, the most popular organisations included some of GiveWell’s top-rated charities, all of which were focused on global poverty. Once again, AMF received by far the most in total donations in both 2015 and 2016. GiveWell, despite only attracting the fourth highest number of individual donors in both 2015 and 2016, was second in terms of amount per donation received each year.

Meta organisations were the third most popular cause area, in which CEA was by far the most favoured in terms of number of donors and combined size of donations in both years. Mercy for Animals was the most popular out of the animal welfare organisations in both years in number of donors, though the Good Food Institute received more in donations than MFA in 2016. MIRI was the most popular organisation focusing on the far future, which was the least popular cause area overall by donation amount (though the fact that only two far future organisations were listed may explain this, at least in part). However, the least popular organisations among EAs were spread across cause areas: Sightsavers and The END Fund were the two least popular, followed by Faunalytics, the Foundational Research Institute and the Malaria Consortium. The relative unpopularity of Sightsavers, The END Fund and the Malaria Consortium, despite their focus on global poverty, may relate to the fact that they were only confirmed on GiveWell’s list of top-recommended charities quite recently and are not in GiveWell’s default recommendation for individual donors.

The results solely for the 476 GWWC members in the sample were similar to the above. Global poverty was the most popular cause area, with ~41% respondents reporting to having donated to organisations within this category. This was followed by cause-prioritization organisations, to which ~13% donated.

Top Donation Destinations

For both 2015 and 2016, the survey results suggest that GiveWell had the largest mean donation size ($5,179.72 in 2015 and $6,093.822 in 2016). Therefore, despite receiving far fewer individual donations than AMF, the total of GiveWell’s combined donations in both years was almost as large. Nevertheless, AMF had the second largest mean donation size ($2,675.39 in 2015 and $3,007.63 in 2016) followed by CEA ($2,796.66 in 2015 and $1,607.32 in 2016). Although GiveWell and CEA were not among the top three most popular organisations for individual donors, they were, like AMF, the most popular within their respective cause areas.

The top twenty donors by donation size in 2016 donated similarly to the population as a whole. The top twenty donors donated the most to poverty charities, and specifically AMF within that cause area. However, the third most popular organisation among these twenty individuals was CEA, which was not one of the top five highest-ranked organisations in aggregate donations for either 2015 or 2016.

Credits

Post written by Eve McCormick, with edits from Tee Barnett and analysis from Peter Hurford.

A special thanks to Ellen McGeoch, Peter Hurford, and Tom Ash for leading and coordinating the 2017 EA Survey. Additional acknowledgements include: Michael Sadowsky and Gina Stuessy for their contribution to the construction and distribution of the survey, Peter Hurford and Michael Sadowsky for conducting the data analysis, and our volunteers who assisted with beta testing and reporting: Heather Adams, Mario Beraha, Jackie Burhans, and Nick Yeretsian.

Thanks once again to Ellen McGeoch for her presentation of the 2017 EA Survey results at EA Global San Francisco.

We would also like to express our appreciation to the Centre for Effective Altruism, Scott Alexander via SlateStarCodex, 80,000 Hours, EA London, and Animal Charity Evaluators for their assistance in distributing the survey. Thanks also to everyone who took and shared the survey.

The annual EA Survey is a volunteer-led project of Rethink Charity that has become a benchmark for better understanding the EA community. This post is the second in a multi-part series intended to provide the survey results in a more digestible and engaging format. Important to bear in mind is the potential for sampling bias and other considerations outlined in the methodology post published here. You can find key supporting documents, including prior EA surveys and an up-to-date list of articles in the EA Survey 2017 Series, at the bottom of this post.

Summary

EAs remain predominantly young and male, though there has been a small increase in female representation since the 2015 survey.

The top five cities with the highest concentration of EAs include the San Francisco Bay Area, London, New York, Boston/Cambridge, and Oxford.

The proportion of EA’s that identify as atheist, agnostic, or non-religious came down from 87% in the 2014 and 2015 surveys to 80% in the 2017 survey.

The number who saw EA as a moral duty or opportunity increased, and the number who saw it as an only an obligation decreased.

Age

The EA community is still predominantly represented by a young adult demographic, with 81% of those giving their age in the EA survey falling between 20 and 35 years of age[1]. This year, ages ranged between 15 to 77, with a mean age of 29 and a median age of 27 (and a standard deviation of 10 years). The histogram below shows a visual representation of the distribution of ages.

[1] Ages were calculated by subtracting the self-reported birth year from 2017.

Gender

The survey respondents were male by a wide majority. Of the 1,080 who answered the question asking how they self-identified regarding gender, 757 (70.1%) identified as male, 281 (26.01%) identified as female, 21 (1.9%) respondents identified as “other”, and another 21 respondents preferred not to answer. This is similar to the 2015 survey, which had a 73% proportion of males.

Consistent with the results of the previous survey, the US and UK are main hubs for EA, home to the majority (63.4%) of this year’s surveyed EAs. Additionally, the top five countries by population (US, UK, Germany, Canada, and Australia) from the 2015 survey remain the top five countries again in 2017. Australia and New Zealand both dropped ranking slightly, and we saw a small increase of EAs living in Northern European countries, such as Germany, Denmark, Sweden, the Netherlands, and the Czech Republic. Representation from Continental Europe overall rose from 14% to 18%.

The San Francisco Bay Area (which includes Berkeley, San Francisco, Oakland, Mountain View, Menlo Park, and other areas) remains the most populous area for EAs in our survey for this question, but only outnumbers respondents from London by a very small margin. This gap between London and the Bay Area has shrunk substantially from 2015.

Oxford, Boston/Cambridge (US) and Cambridge (UK) all show consistently high populations of EAs. Washington D.C. dropped from the fifth most densely populated EA city to eleventh. Newly reported additions include Berlin, Sydney, Madison, Oslo, Toronto, Zürich, Munich, Philadelphia, and Bristol.

The proportion of atheist, agnostic or non-religious people is less than the 2015 survey. Last year that number was 87% compared to 80.6% this year. That metric hadn’t changed over the last two surveys, so this could be an indicator that inclusion of people of faith in the EA community is increasing.

As noted in 2015, it has been suggested that greater efforts should be made on the part of EA to be more inclusive of religious groups. The numbers definitely still show room for growth in religious communities.

The distribution of responses regarding a stance on moral philosophy is extremely similar to the last survey. In 2015, 56% selected Consequentialism (Utilitarian), 22% No opinion or not familiar with these terms, 13% Non-utilitarian consequentialism, 5% Virtue Ethics and 3% Deontology. Among respondents, the distribution of philosophical stances has not noticeably changed.

Do they see EA as an opportunity or an obligation?

This question was inspired by Peter Singer’s classic essay on whether doing a tremendous amount of good is an obligation or an opportunity, which inspired commentary by Luke Muehlhauser (see this post) and Holden Karnofsky (see this post), among others. Perhaps even more than a preferred moral philosophical stance, this helps us get a view to the participants’ motivation to be effective altruists.

The 2015 survey posed this question a little differently, presenting the choices as ‘Opportunity,’ ‘Obligation,’ or ‘Both’ instead of ‘Moral Duty’. Both surveys included ‘Other’ as a choice as well. About the same proportion chose ‘Both’ in 2015, as those who selected ‘Moral Duty’ this year. We could guess that there was a richer connotation understood by ‘Moral Duty’, over the more narrow, and somewhat negatively biased ‘Obligation’ option.

From 2015 to this year, those who saw EA as only an opportunity stayed the same, while those seeing it only as an obligation decreased significantly.

By offering ‘Moral Duty’ as a response, we may have given those who see participating in EA as primarily a dutiful action, a more neutral (less negative) and/or more principled (less self-focused) match to their personal interpretation.

Credits

Post written by Katie Gertsch, with edits from Tee Barnett and analysis from Peter Hurford.

A special thanks to Ellen McGeoch, Peter Hurford, and Tom Ash for leading and coordinating the 2017 EA Survey. Additional acknowledgements include: Michael Sadowsky and Gina Stuessy for their contribution to the construction and distribution of the survey, Peter Hurford and Michael Sadowsky for conducting the data analysis, and our volunteers who assisted with beta testing and reporting: Heather Adams, Mario Beraha, Jackie Burhans, and Nick Yeretsian.

Thanks once again to Ellen McGeoch for her presentation of the 2017 EA Survey results at EA Global San Francisco.

The annual EA Survey is a volunteer-led project of Rethink Charity that has become a benchmark for better understanding the EA community. This post is the first in a multi-part series intended to provide the survey results in a more digestible and engaging format. You can find key supporting documents, including prior EA surveys and an up-to-date list of articles in the EA Survey 2017 Series, at the bottom of this post.

Platform and Collection

Data was collected using LimeSurvey. This year, a “Donations Only” version of the survey was created for respondents who had filled out the survey in prior years. This version was shorter and could be linked to responses from prior years if the respondent provided the same email address each year.

Distribution Strategy

Author Note: Any mention of distribution of “the survey” refers to the URL of the full effective altruism (EA) survey as well as the URL for the “Donations Only” version of the survey. Each URL has a unique tracking tag that referenced the organization or group sharing the URLs and the type of medium it was being shared on. For example, the URLs shared in the 80,000 Hours newsletter had the tracking tag “80k-nl”.

Distribution began on April 19, 2017 and continued on a rolling basis until the close of the survey on June 16, 2017. The expansive outreach plan and lag time associated with particular forms of outreach necessitated distributing the survey on a rolling basis. We reached out to over 300 individuals and groups that posted the survey on our behalf, and/or who required permission by a group administrator for a member of the survey team to post the link to a specific site.

To minimize undersampling and oversampling of different parts of EA, and to make the survey as representative of the community as a whole, we initially followed the distribution plan from the 2014 and 2015 EA surveys, and made modifications based on team consensus. This distribution plan was implemented in 2014 by Peter Hurford, Tom Ash, and Jacy Reese to reach as many members of the EA population as possible. Certain additions and omissions were made depending on the availability of particular channels since the initial drafting of the distribution plan. Anyone who had access to the survey was encouraged to share it.

An appropriate amount of caution should accompany any interpretation of the EA survey results. While the distribution plan included all known digital avenues to reach the EA population, there is room for error and bias in this plan. Claims that a certain percentage of respondents to the survey have certain predispositions or harbor certain beliefs should not necessarily be taken as representative of all EAs or “typical” of EAs as a whole. Any additional suggestions on how to reach the EA community are welcome.

In an attempt to maximize community engagement, we distributed the survey through email mailing lists, the EA slack group, social networks, forums, websites, emailing prior survey takers, and personal contact.

The survey was shared on the following websites and forums:

The survey team reached out to the following mailing lists and listservs to share the survey, those with an asterisk confirmed that they had shared the survey:

The survey was posted to the following general Facebook groups:

The survey was shared with the following local and university Facebook groups, it might not have been posted to all groups due to permissions from administrators:

The survey was also emailed to those who had taken the 2014 and/or 2015 survey and had provided their email address.

Data Analysis

Analysis began on June 16, 2017 when the dataset was exported and frozen. Any responses after this date were not included in the analysis. The analysis was done by Peter Hurford with assistance from Michael Sadowsky.

Analysis was done in R. All scripts and associated data can be found in the public GitHub repository for the project (see the repository here and the anonymized raw data for the 2017 survey here). Data was collected by Ellen McGeoch and then transferred to the analysis team in an anonymized format, as described in the survey’s privacy policy. Currencies were converted into American dollars and standardized, and then processed and analyzed using the open source Surveytools2 R package created by Peter Hurford.

Subpopulation Analysis

In general, people found our survey via Facebook (such as the main EA Facebook group, but not including Facebook pages for local groups), SlateStarCodex, local groups (mailing lists and Facebook groups), the EA Forum, the EA Newsletter, people personally sharing the survey with others, LessWrong, Animal Charity Evaluators (social media and newsletter), 80,000 Hours (newsletter), and an email sent to prior survey takers.

By numbers, the referrers broke down like this:

Referrer data was gathered via URL tracking. We also asked people to self-report from where they heard about the survey. Similar to the 2014 and 2015 surveys, the self-report data does not line up with the URL data perfectly (e.g., only 72.73% of those for whom URL tracking shows they took it from the EA Newsletter said they heard about the survey from the EA Newsletter). While we don’t know the cause of this, one possible reason might be that some individuals first hear of the survey from one source, but don’t actually take it until they see it posted via another source. Given this discrepancy, we consider URL tracking to be more reliable for determining referrers.

Since we know what populations we are drawing from, we want to know two key questions:

Do our subpopulations successfully capture EA as a whole? If we have 2.2% (19 LessWrong refers divided by 856 people who responded) of our population coming from LessWrong, is this close to the “true” number of self-identified EAs that frequent LessWrong more than other channels? Are we over- or under-sampling LessWrong or other channels? Are we systematically missing any part of EA by not identifying the correct channels in order to get people to respond?

Do we successfully capture our subpopulations? Are the people who take the survey from LessWrong actually representative of EAs who frequent LessWrong more than other channels? Are we systematically misrepresenting who EAs are by getting a skewed group of people who take our survey?

Do our subpopulations successfully capture EA as a whole?

Unfortunately, we can’t answer this question outright without knowing what the “true” population of EAs actually looks like. However, we can evaluate the strength of that concern by seeing how different our subpopulations are from each other. If our subpopulations vary substantially, then oversampling and undersampling can dramatically affect our representativeness. If our subpopulations don’t vary by a large margin, then there is less risk from undersampling or oversampling individual populations that we did sample from, but there is still risk from missing populations that we did not sample.

Based on the above table, it seems our subpopulations do differ in demographics and affinity toward causes, but not in donation amounts or income. There is a definite risk that oversampling some groups and undersampling others could introduce bias in demographics and answers like top causes.

As a contrived example to demonstrate what this bias could look like, imagine that SSC truly has 500 EAs on the site all of which are entirely male, and 400 of them take our survey. Whereas, the EA FB group has 1000 EAs, is entirely female, but only 100 of them take our survey. This means that the “true” population (in our contrived example) would be 33% male, whereas our sampled population would be 80% male.

Unfortunately, without knowing the true distribution of EAs, there’s no real way we can know whether we oversampled, undersampled, or got things close to right. This means we should be careful when interpreting EA survey results.

Do we successfully capture our subpopulations?

The next question is how well we capture our subpopulations. Again, without an unbiased census of the entire subpopulation, it will be difficult to tell. However, we can compare to another survey. We did some detailed analysis on this for the 2014 EA Survey. There haven’t been that many other surveys of EAs lately, but there was a 5500 person survey of SlateStarCodex readers launched just two months before we launched our survey.

The SSC Survey had many more SSC readers who were EAs than our EA Survey had EA Survey takers who are SSC readers. However, it seems that our EA Survey properly matched the SSC Survey on many demographics, with the exception that the EA Survey had a more consequentialist audience that donated slightly more while earning slightly less. This would indicate that there is a good chance we adequately captured at least the SSC survey-taking EA population in our EA Survey.

Credits

Post written by Ellen McGeoch and Peter Hurford, with edits from Tee Barnett and analysis from Peter Hurford.

A special thanks to Ellen McGeoch, Peter Hurford, and Tom Ash for leading and coordinating the 2017 EA Survey. Additional acknowledgements include: Michael Sadowsky and Gina Stuessy for their contribution to the construction and distribution of the survey, Peter Hurford and Michael Sadowsky for conducting the data analysis, and our volunteers who assisted with beta testing and reporting: Heather Adams, Mario Beraha, Jackie Burhans, and Nick Yeretsian.

Tags

"Rethink Charity" is not represented by a single organization, but by several non-profit organizations in various countries, including Rethink Charity USA (EIN# 82-5325150) and the Students for High-Impact Charity Foundation of Canada (BN 75152 2293 RC0001). Rethink Charity also receives support from Effective Altruism UK (charity number 1170614).