Critical Algorithm Studies: a Reading List

This list is an attempt to collect and categorize a growing critical literature on algorithms as social concerns. The work included spans sociology, anthropology, science and technology studies, geography, communication, media studies, and legal studies, among others. Our interest in assembling this list was to catalog the emergence of “algorithms” as objects of interest for disciplines beyond mathematics, computer science, and software engineering.

As a result, our list does not contain much writing by computer scientists, nor does it cover potentially relevant work on topics such as quantification, rationalization, automation, software more generally, or big data, although these interests are well-represented in these works’ reference sections of the essays themselves.

This area is growing in size and popularity so quickly that many contributions are popping up without reference to work from disciplinary neighbors. One goal for this list is to help nascent scholars of algorithms to identify broader conversations across disciplines and to avoid reinventing the wheel or falling into analytic traps that other scholars have already identified. We also thought it would be useful, especially for those teaching these materials, to try to loosely categorize it. The organization of the list is meant merely as a first-pass, provisional sense-making effort. Within categories the entries are offered in chronological order, to help make sense of these rapid developments.

In light of all of those limitations, we encourage you to see it as an unfinished document, and we welcome comments. These could be recommendations of other work to include, suggestions on how to reclassify a particular entry, or ideas for reorganizing the categories themselves. Please use the comment space at the bottom of the page to offer suggestions and criticism; we will try to update the list in light of these suggestions.

0. overviews0.1 technical and philosophical precursors / emic “what are algorithms?” essays0.2 field surveys / keywords / initial provocations0.3 books about algorithms addressed to broader audiences0.4 conferences focused on algorithms and society0.5 lists of algorithm studies resources0.6syllabi that focus on algorithms and society
1. the specific implications of algorithms and the choices they make1.1 algorithms have embedded values / biases, lead to personalization / social sorting / discrimination1.2 with algorithms come rationalization / automation / quantification, and the erasure of human judgment / complexity / context1.3 questions of accountability and policy responses around algorithms2. algorithms fit with, and help advance, specific ideological worldviews3. algorithms are complex technical assemblages, that have to be mapped4. algorithms aren’t just technical artifacts, they’re fundamentally human in their design and their use4.1 people design and maintain algorithms, in specific ways, and that matters4.2 people work, play, and live with algorithms, in specific ways, and that matters4.3 what do users understand about algorithms4.4 the discursive production of algorithms to shape their public perception5. methods and approaches for studying algorithmic systems

Ziewitz, Malte. 2011. How to think about an algorithm? Notes from a not quite random walk. Discussion paper for Symposium on “Knowledge Machines between Freedom and Control”, 29 September. http://zwtz.org/files/ziewitz_algorithm.pdf

Hallinan, Blake, and Ted Striphas. 2014. “Recommended for You: The Netflix Prize and the Production of Algorithmic Culture.” New Media & Society, 18(1), 117-137. http://nms.sagepub.com/content/18/1/117

Webmoor, Timothy. 2014. “Algorithmic Alchemy, or the Work of Code in the Age of Computerized Visualization.” In Annamaria Carusi, Aud Sissel Hoel, Timothy Webmoor, and Steve Woolgar, eds., Visualization in the Age of Computerization. 19-39. New York: Routledge.

Beunza, Daniel and Millo, Yuval. 2015. “Blended automation: integrating algorithms on the floor of the New York Stock Exchange” SRC Discussion Paper No. 38. London School of Economics and Political Science. http://eprints.lse.ac.uk/65090/

Rieder, Bernard, and Sire, Guillaume. 2014. “Conflicts of Interest and Incentives to Bias: A Microeconomic Critique of Google’s Tangled Position on the Web.” New Media & Society 16 (2): 195–211. http://nms.sagepub.com/content/16/2/195

Gehl, Robert W. 2014. Reverse Engineering Social Media: Software, Culture, and Political Economy in New Media Capitalism. Philadelphia, Pennsylvania: Temple University Press.

This is a superb point, and thanks so much for all of the useful links. nick and I had a similar question when it came to literature that gathers around the idea of “big data,” which we might consider either the same literature, or certainly a close sibling. We decided to focus our efforts on work that explicitly orients itself towards algorithms, trying to work out what that should mean sociologically, and that uses it as a analytical step. But I think a larger, umbrella list might include scholarship focused on algorithms, scholarship focused on artificial intelligence, scholarship focused on machine learning, scholarship focused on big data, and scholarship focused on complex computational systems. Perhaps we need five lists that are themselves components of a mega list. Thanks again!

Graham, M., M. Zook., and A. Boulton. 2013. Augmented Reality in the Urban Environment: contested content and the duplicity of code. Transactions of the Institute of British Geographers. 38(3), 464-479.

I’m very excited about this list, looks like some interesting reading.

In case you’re interested, I figured I’d mention our work, as I think it fits particularly well into section 4.2:

Jacob Thebault-Spieker, Loren G. Terveen, and Brent Hecht. 2015. Avoiding the South Side and the Suburbs: The Geography of Mobile Crowdsourcing Markets. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW ’15). ACM, New York, NY, USA, 265-275. DOI=http://dx.doi.org/10.1145/2675133.2675278

Thank you so much for compiling this list. I think it will be extraordinarily helpful to my work!

I am fascinated by the disciplinary distinction you are drawing here between engineering/CS/etc. and the areas of scholarship that you are including. As we know, where these kinds of boundaries are drawn matter. Why this boundary?

Thanks! The distinction we meant, at least at first, was that we were specifying a more sociological attention to algorithms and their implications, as opposed to the vast literature about algorithms and their workings, i.e. the CS scholarship that focuses on designing them. This is not meant to exclude literature, for example in ACM contexts, that are nevertheless asking political, sociological, or anthropological kinds of questions. It is admittedly a fuzzy distinction, and I for one am ill-equipped to be able to specify the difference inside of CS-styled scholarship.

A great list I am recommending to my students – I wonder whether it could be turned into a downloadable RIS file or Mendeley group or similar to make it easy to “drill down” to the abstracts (or keyword search them). Or even just put the list of headings at the top, anchored to the headings down below?

Halfaker, A., Geiger, R. S., Morgan, J. T., & Riedl, J. (2012). The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline. American Behavioral Scientist, 0002764212469365.https://www-users.cs.umn.edu/~halfak/publications/The_Rise_and_Decline/halfaker13rise-preprint.pdf
– Discusses how Wikipedia editors developed technologies that use subjective algorithms to make quality control more efficient, but they inadvertently also made Wikipedia a harsh place for newcomers.

Halfaker, A., Geiger, R. S., & Terveen, L. G. (2014, April). Snuggle: Designing for efficient socialization and ideological critique. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 311-320). ACM.http://www-users.cs.umn.edu/~halfak/publications/Snuggle/halfaker14snuggle-preprint.pdf
– Discusses how the standpoint of the designers of Wikipedia’s algorithmic quality control tools was enacted through the processes that the tools support. Also describes a proof of concept system that uses the same subjective algorithms to reverse the order and open up the more dominant systems to critique.

Margetts, H., & Dunleavy, P. (2013). The second wave of digital-era governance: a quasi-paradigm for government on the Web. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 371.

Are you taking suggestion for removals? I think having the Kowalski piece at the top of ( overviews | technical ) is pretty misleading. Kowalski is arguing for the use of predicate logic to **analyze** algorithms, but the piece is written with an overbroad title and abstract that means it could easily be mistaken for a definition of algorithms if you don’t read it all the way through. Far from being an attempt to define the term “algorithm” or a foundational article, it’s an obscure approach that didn’t catch on.

Abstract: This paper outlines the notion of ‘algorithmic technique’ as a middle ground between concrete, implemented algorithms and the broader study and theorization of software. Algorithmic techniques specify principles and methods for doing things in the medium of software and they thus constitute units of knowledge and expertise in the domain of software making. I suggest that algorithmic techniques are a suitable object of study for the humanities and social science since they capture the central technical principles behind actual software, but can generally be described in accessible language. To make my case, I focus on the field of information ordering and, first, discuss the wider historical trajectory of formal or ‘mechanical’ reasoning applied to matters of commerce and government before, second, moving to the investigation of a particular algorithmic technique, the Bayes classifier. This technique is explicated through a reading of the original work of M. E. Maron in the early 1960 and presented as a means to subject empirical, ‘datafied’ reality to an interested reading that confers meaning to each variable in relation to an operational goal. After a discussion of the Bayes classifier in relation to the question of power, the paper concludes by coming back to its initial motive and argues for increased attention to algorithmic techniques in the study of software.

About Us

The Social Media Collective (SMC) is a network of social science and humanistic researchers, part of the Microsoft Research labs in New England and New York. It includes full-time researchers, postdocs, interns, and visitors. Beginning in 2009, the researchers who now lead the initiative are: Nancy Baym, danah boyd, Kate Crawford, Tarleton Gillespie, and Mary Gray. Our primary purpose is to provide rich contextual understanding of the social and cultural dynamics that underpin social media technologies. We use a variety of methodologies and span multiple disciplines.

Subscribe by Email!

Enter your email address to subscribe to the SMC blog. You'll get an email when we update the blog.