Undercover Investigations

Animal advocacy organizations use undercover1 investigations, sometimes just referred to as investigations, to document animal abuse and raise awareness of various forms of suffering and injustice against animals. This increased awareness is expected to result in various outcomes that benefit animals, including influencing company policies, laws, and consumption choices. We focus our research on investigations of animal agriculture,2 but this intervention is also used to document animal abuse in other areas, such as laboratory research that involves live animals3 and the breeding of companion animals.4 We believe that undercover investigations have had significant direct effects in reducing farm animal suffering through corporate policy change. We think they also have major effects, in expectation,5 on consumption choices and some long-term outcomes such as increasing the likelihood of future corporate policy change, although the size of long-term effects is much more uncertain. While there’s a chance that some undercover investigations could have bad long-term outcomes, we think that risk is relatively small.

What are undercover investigations?

We use the term “undercover investigation” to refer to any project where activists obtain documentation (e.g. photos and videos) of the treatment of animals without the explicit cooperation of the people or organizations using the animals. We focus primarily on investigations of farms and other animal agriculture facilities. This documentation is usually shared with the public through mainstream news and social media.6 Although the evidence obtained is sometimes used to press charges, the documented abuses are often legal and standard industry practice.7 Organizations present the evidence from these investigations alongside messages that advocate for the end of animal agriculture, advocate for less suffering of farmed animals (but not opposing animal agriculture itself), or a blend of the two.

Investigations are often connected to other forms of advocacy. The documentation and public outrage resulting from the investigation is often used by the investigating organization to pressure the company supplied by the facilities being investigated to improve its animal welfare policies.8 The documentation is also sometimes used in grassroots and online outreach intended to inspire consumers to reduce or eliminate their consumption of animal products.9

What are their strengths?

In the best-case scenario, undercover investigations can lead to immediate changes through corporate or legal policy change and consumers reducing their consumption of animal products, as well as contribute to long-term outcomes by growing the animal advocacy movement and increasing public concern for farm animal issues.

Investigations seem to gather a significant amount of attention in the news and on social media.10 There is also clear evidence that they have led to corporate policy change.11 We are much more uncertain what effects they have on consumer choices and long-term change, although we expect both are fairly promising.

We are also quite uncertain about what factors make an investigation most effective. Some factors, like whether an individual worker is criminally charged or whether the focus of the investigation is on blatant abuse like punching and kicking animals, might increase the amount of attention an investigation receives, but also might reduce the impact of the publicity because (i) this abuse seems more easily dismissed as “one bad farm” or “one bad worker,” and (ii) the issue being highlighted constitutes a smaller portion of the the total suffering farmed animals endure compared to practices like extreme confinement and chronic health issues.12, 13

In addition to focusing on the most severe forms of suffering, we think that investigations can be more effective by focusing on individual stories. This can involve combining footage of various facilities to show the life story of the farmed animals. It can also involve highlighting the work of a specific investigator, especially when they discuss how the investigation affected them personally. We generally believe storytelling is an effective way to promote empathy and connect people with farm animal issues.14

It seems like some factors likely increase the impact of investigations in specific situations. For example, when a certain unethical practice, such as confining hens in battery cages, is being critiqued in popular media, investigations that highlight this issue could synergize with existing pressure to change policies or gain additional attention.15

What are their weaknesses?

Although we have seen substantial impact from investigations in terms of media coverage and corporate policy change, there’s less evidence for their potential impact on consumer choices and long-term outcomes. One concern, mentioned above, is that some investigations might give the impression that the issues with the animal agriculture industry are limited to a few abusive workers. Although we recognize this concern, we think that current media coverage indicates that the public recognizes more pervasive issues, such as the confinement of animals in battery cages and gestation crates.16

The undercover investigations that achieve the most coverage in the U.S. are those that involve investigators gaining employment at animal agriculture facilities.17 These investigations are quite expensive compared to strategies popular in other countries like visiting the facility at night and placing hidden cameras to document abuse.18 Still, even employment-based investigations seem to be a cost-effective way of capturing media coverage.19

It appears that, at least in the U.S., undercover investigations have diminishing marginal returns in their ability to capture media coverage after a certain point. Mercy for Animals capped the number of investigations released in 2015 for this reason,20 and are working on expansion into other countries where saturation is less of a concern.21 There are many countries where investigations have not become very popular, and as long as one believes work in these countries is not much less effective, this means additional funding likely has similar effects as existing funding in undercover investigations. Additional funding might have greater effects in some countries due to lower costs.

There’s a plausible concern that this saturation suggests all future investigations are less effective, rather than just investigations beyond the current rate. We aren’t very concerned about this because media outlets appear to be at least as interested in covering investigations as they used to be as long as there are not too many being released around the same time, and further investigations make the average reader/viewer more likely to realize that the issues are systemic across the animal agriculture industry.

Overall, we think the weaknesses of undercover investigations are limited.

What about long term effects?

Possible Positive Effects

The long term effects of undercover investigations are quite unclear, and we have little evidence available here beyond speculative reasoning.

Similarly to corporate outreach, we think investigations can promote discussion and concern for the treatment of farmed animals. In the long term, this can increase the chances of success for other advocacy efforts, such as leafleting and ads, corporate policy change, and legal change. The current perception in the U.S. that farmed animals are frequently mistreated for commercial gain,22 especially through extreme confinement practices, might be the result, at least in part, of the many undercover investigations released over the past several years.23 The outreach methods like leafleting and online ads that use undercover investigations might also be a significant factor in this change.

Possible Negative Effects

As with possible positive long term effects, we have much uncertainty about the possible negative long term effects.

The footage used in undercover investigation videos could make people depersonalize the individual animals affected, especially if the rhetoric of the video is abstract and generalized rather than speaking of the animals as individuals.24 It’s possible that incorporating “happy” footage of animals, like that from farm animal sanctuaries, could reduce this effect if it exists.

Many investigations focus on blatant abuse by individual workers, such as punching and kicking the animals.25 They also tend to target a specific farm.26 This could risk making people think the issues of animal agriculture are localized or exist at only a few “bad apple” farms. We think this concern is reduced when viewing undercover investigations in aggregate. Given how many have come out, and that advocate rhetoric emphasizes the systemic nature of the abuse, we don’t think this is a significant concern.

Investigations frequently involve deceit, such as working at the farm while wearing hidden cameras. Some investigations that don’t involve deceit involve visiting farms without the permission of the facility’s owners.27 Either of these approaches could lead to a negative view of animal advocates if they are seen as liars or criminals. On the other hand, they could also make advocates seem brave or make the cause seem more important since advocates are willing to go to such lengths to help the animals, which seems to be more common in public discussion of investigations to date.28

Conclusions

Overall, we think investigations fit well into long term animal advocacy strategy. It seems more likely than not that investigations increase concern for farmed animals, and that outcome seems very promising for building a better world for animals. There are plausible concerns that some investigations could negatively affect the way people perceive animal abuse, such as by suggesting that it is a localized concern that can be easily resolved. Like with other animal advocacy interventions, we are much more uncertain about the long-term effects than the short-term effects. We would tentatively conclude that the long-term effects of investigations seem more promising than other interventions like leafleting and corporate outreach, but we put limited weight on this consideration in our overall understanding of how effective or ineffective any intervention is.

Do we recommend it? Why or why not?

We recommend undercover investigations in most instances when done by experienced organizations, although we are concerned about saturation in the U.S. that could reduce the effectiveness of investigations beyond the current level, and the potential for investigations that are conducted poorly to reduce the overall reputation of animal advocates. We think that more research is needed on what makes some investigations more impactful than others, but tentatively, we consider thoughtful messaging, such as emphasizing that abuse is rampant throughout the industry and highlighting the stories of individual animals, as important. Additionally, undercover investigations seem particularly effective when coupled with other advocacy strategies, such as protests and corporate outreach, especially if they provide public pressure for a specific campaign, such eliminating battery cages.

What are characteristics of a strong undercover investigation?

Coverage of the investigation reaches a wide audience, usually by being featured in a major news outlet and promoted on social media. This increases the number of people who may change their consumption as a result of the investigation, participate in activism, or help animals in other ways. It also increases the credibility of the investigation, which can make individuals and institutions, such as animal agriculture companies, take the investigation more seriously. We are fairly uncertain about what increases coverage of an investigation, but some promising factors include connections from the farm being investigated to well-known companies and the existing media contacts of the organization conducting the investigation.

The investigation is conducted with careful consideration for how it will be perceived by the general public, limiting the risk of harm from advocates being seen as inappropriately deceitful, criminal, or having other negative features and the risk of investigators being charged with a crime, including associated legal fees. This is especially important because it could affect the perception of investigation groups in general.

The investigation is conducted with a thoughtful understanding of the current landscape of animal advocacy, including which practices or companies are currently under heightened scrutiny. By incorporating that focus, such as investigating a battery cage facility while battery cage bans are being discussed in the national media, an investigation can both gain increased attention and make other efforts more likely to succeed, such as legal and corporate reform.

The messaging used when the investigators present their documentation is optimized to effect change, such as by highlighting individual animals and systemic abuse. Highlighting the fact that the mistreatment of animals is an institutional issue, rather than a personal or localized issue, might do more to inspire positive action such as dietary change and activism. A systemic focus also aligns investigating organizations with other groups who advocate on behalf of workers in the animal agriculture industry. These workers often suffer from physical hazards, difficulty in earning a living wage, and other issues.29

The investigation is conducted by an organization with prior experience in this area, or the leaders of the project have discussed possible strategies with an experienced organization. There are many organizations with strong records in undercover investigations, and advocates should use their experience to inform future campaigns. Some organizations that have had success with undercover investigations are Mercy for Animals, Animal Equality, Compassion Over Killing, Direct Action Everywhere, The Humane Society of the United States, and People for the Ethical Treatment of Animals. Each of these groups uses somewhat different tactics.

How strong is the evidence about the efficiency of undercover investigations?

The primary immediate effects of undercover investigations ― corporate policy change and media coverage ― are directly observable, although the extent to which policy changes are attributable to an investigation as opposed to corporate outreach is unclear.

The other effects of undercover investigations, such as reductions in consumption of animal products, have less robust evidence. The most scientific piece of evidence we know of is an economics study that found, “Increasing media attention to animal welfare issues triggers consumers to purchase less meat rather than reallocate expenditures across competing meats,” based on an analysis of national consumption data and numbers of new stories between 1999 and 2008.30 Our report on this study, and a similar study on media attention and demand for eggs, found them to be inconclusive.31 We will also consider ways that we can conduct similar analyses of our own on the relationship between media coverage and consumer purchases.

Given the limited evidence on the effects of undercover investigations on dietary change and other indirect outcomes, we are uncertain but quite optimistic about the total impact of this intervention.

We primarily focus on investigations that involve going “undercover,” meaning without the facility owner’s explicit consent to document the facilities and use that documentation to spread awareness of the problems involved. Some investigations, but not many, do involve explicit cooperation, such as when a chicken farmer working for Perdue Farms cooperated with Compassion in World Farming to document what the farmer saw as inappropriate farming practices. See McKenna, M. (February 19, 2015). A Factory Farmer Strikes Back at the Company He Works for. Wired.

Our claims in this review apply similarly to different types of investigations, including employment-based, open rescues, and cooperative. We have noted where our beliefs differ depending on the type of investigation.

HSUS is one organization that conducts undercover investigations of live animal research facilities.

“A nine-month undercover investigation by The Humane Society of the United States (HSUS) has pulled back the curtain on the secretive, federally-funded New Iberia Research Center (NIRC) in Louisiana, revealing routine and unlawful mistreatment of hundreds of chimpanzees and other primates.”—(March 4, 2009). Undercover Investigation Reveals Cruelty to Chimps at Research Lab. HSUS.

PETA is one organization that conducts undercover investigations of “puppy mills.”

“A PETA investigator worked for months at Nielsen Farms, a puppy mill in Kansas. The investigator’s job was to feed, water, and clean up after hundreds of dogs condemned to cramped wire enclosures.”—Puppy Mill Prison. PETA.

By “in expectation,” we mean “in terms of expected value,” which is the product of the likelihood of the effect and the magnitude of the effect if it occurs. For example, we would say taking a lottery where you have a 10% likelihood of earning a million dollars has a major effect on your total income, in expectation, even though the most likely outcome is that you will not win the lottery.

“MFA also reaches out to mainstream media outlets that might be interested in covering the investigation.

…

Mainstream media is not the only way that MFA distributes their work; they also use social media.”—Conversation with Matt Rice (February 9, 2015).

“Matt, MFA’s legal team, and independent animal welfare experts then review relevant portions of the footage and evaluate whether what it shows is criminal abuse or standard industry practice (which is also abusive). When criminal abuse is identified, Matt works with MFA’s legal team to draft formal complaints and get them to appropriate law enforcement agencies.”—Conversation with Matt Rice (February 9, 2015).

“MFA’s investigation team also works to identify all the links in the facilities supply chain, from the farm to the point of sale for consumers, so MFA can work with companies in the supply chain to make animal welfare policy changes.”—Conversation with Matt Rice (February 9, 2015).

It seems most media coverage of the suffering of animals in animal agriculture is in response to undercover investigations. The other common topic is corporate policy change, such as switching to cage-free eggs. Other forms of activism like leafleting and protests seem to get less media coverage relative to the amount of resources put into them, although we have not done a systematic analysis of the differences.

For example, Animal Equality estimates that their investigations have earned 182 million media views in 2015 and 312 million in 2014.

For example, “The investigation at Wiese Brothers Farms led to one of their largest corporate outreach victories. This was a dairy farm that they connected to DiGiorno Pizza, which is owned by Nestle, the largest food company in the world. Nestle got such a large media and public response that they decided to work with MFA to implement an animal welfare policy, and the policy they implemented is probably the most comprehensive animal welfare policy ever implemented by a company. They announced a policy change in 2014 that will apply to every farm that supplies them, in 90 countries.”—Conversation with Matt Rice (February 9, 2015).

One example of an investigation that focused on blatant worker abuse was a recent video released of a Perdue chicken supplier. News coverage of this investigation mostly focused on the individual worker’s actions, such as this piece, “Man arrested after undercover video reveals alleged abuse at Perdue chicken supplier,” in the Washington Post. The quotes from animal advocates in the piece did refer to the systemic nature of abuse in the industry, which we see as increasing the impact of the coverage.

Although we do not have a formal write-up on which abuses of farmed animals cause the most suffering, and such judgments are necessarily subjective, we think we can make reasonable estimations based on the length the suffering is endured and the severity at each moment. We expect worker abuse, such as punching and kicking, is endured for a much shorter period of time per animal than confinement and health issues, and does not have a proportionally larger severity. We think most animal advocates familiar with these issues would agree with this assessment.

See some of the resources in the Social Psychology section of our Research Library, including “’If I look at the mass I will never act’”: Psychic numbing and genocide” and “Explaining the ‘Identifiable Victim Effect.’”

An example of this strategy was a 2008 investigation of a California egg farm by Mercy for Animals, which was released shortly before California voted on Prop 2, a ballot proposition that prohibits certain kinds of extreme confinement of farmed animals.

In 2015, there was substantial media coverage of battery cages, which frequently referred to undercover investigations and addressed the systemic nature of the issue, such as, “Free the Hens, Costco!” in the New York Times. “Even after an undercover investigation recently documented a Costco egg supplier locking birds in cages with the mummified corpses of their dead cage mates, Costco responded that the supplier was ‘behaving appropriately.’”

The only U.S. group we know of currently doing non-employment-based investigations is Direct Action Everywhere (DXE), which released their first investigations in 2015. The two investigations they have released seem to have received less media coverage than other investigations, although this could change as DXE grows.

Note, the following quote comes from a conversation that took place before the second investigation DXE released, “DXE has had 50 press appearances in 2015 so far, and they expect to get 3-5 per month throughout the year. They estimate that 50-100k people were exposed to animal rights as a result of each of these press appearances. In addition, they place a lot of emphasis on social networks; they had over 50k unique views on their video, and they estimate 5 million people were exposed to their work through social media in the weeks after the initial release, including shared demonstrations and press coverage. This estimate is approximate, since some data is hard to collect with their decentralized model.”—Conversation with Wayne Hsiung (March 13, 2015).

Mercy for Animals is the group that seems to conduct the most investigations in the United States and uses an employment-based strategy. From January to August 2015, they recorded 7,069 media pieces about their work with an estimated billions of people reached with six investigations.

We would note that we have large uncertainty about whether employment-based investigations are actually better at producing media coverage, especially given the small sample size available and confounding variables at play, such as the differing ages and sizes of these organizations. We are not comfortable making a claim in either direction for this effect.

Private communication with a representative of a charity that conducts numerous investigations outside the United States.

“This number is purposefully lower than last year because they were concerned too many investigations could reduce the amount of attention each new one gets in the media.”—Conversation with Nick Cooney (September 2, 2015).

“They now have a full-time investigator in India, gathering footage for them. The first release in India is going to be different than the ones in North America. They also have contracted an investigation firm in China to work in each of the main animal agriculture industries, such as eggs, poultry, pork. In Mexico, MFA has two investigators ready to start working who are Mexican citizens. MFA is finishing legal due diligence to be sure they conduct their investigations within the bounds of state and federal laws.

…

These investigations are especially exciting because there haven’t been many investigations in these countries. This means they are more likely to grab substantial media attention, spark public outrage, and prompt corporate and policy changes.”—Conversation with Nick Cooney (September 2, 2015).

We think most people who work on farm animal advocacy would agree with this claim based on their personal experiences interacting with the public. In general, we haven’t found very convincing survey evidence for or against the claim, but there is one survey that found “A total of 64% of survey respondents agree with the statement, ‘farmers and food companies put their own profits ahead of treating farm animal humanely.’” Lusk, J, Norwood, B.F., Prickett, R. (August 17, 2007). Consumer Preferences for Farm Animal Welfare: Results of a Nationwide Telephone Survey. Oklahoma State University.

Although we haven’t performed a full analysis, it appears that public awareness of poor conditions for farmed animals has increased over the last 10-20 years. For instance, we think there has been more discussion of these issues in the media and through documentaries, and there has been more public reaction through seeking out “humane” alternatives and supporting legal and corporate policies that restrict certain practices. These changes correspond to the growth of undercover investigations as a tactic, and it’s clear how undercover investigations could lead to increased public awareness and that farmers and corporations react to them as if they do.

It seems plausible that if identifiable victims make a problem more personal and personalize the victims, then discussing the issue without identifiable victims could have the opposite effect. However, we have not seen substantial evidence for this effect and we consider it only as a possibility rather than a likely downside.

One example of an investigation that focused on blatant worker abuse was a recent video released of a Perdue chicken supplier. News coverage of this investigation mostly focused on the individual worker’s actions, such as this piece, “Man arrested after undercover video reveals alleged abuse at Perdue chicken supplier,” in the Washington Post. The quotes from animal advocates in the piece did refer to the systemic nature of abuse in the industry, which we see as increasing the impact of the coverage.

Scanning through lists of undercover investigations, such as this one, or lists of news articles covering investigations, we see that most focus on the specific farm involved, often naming them in the headline or mentioning that it was a farm, rather than the industry as a whole, where the abuse occurred.

Direct Action Everywhere conducts this sort of investigation in the United States. When the activists are open about their identities and activities, these are referred to as “open investigations” or “open rescues” (if at least one animal is rescued from the farm by the activists).

One of the largest examples of this framing has been the media and public reaction to “ag gag” laws that have been introducing in state legislatures to prevent undercover investigations from happening. Even media outlets that tend to be less opinionated have still condemned these laws and talked about the importance of undercover investigation work, such as in this interview on CNN Money in June 2015.

For more information on the harms suffered by workers in the animal agriculture industry, see this summary by the Food Empowerment Project.

Regarding the egg study: “With moderately more certainty, we can conclude that Prop 2 media coverage resulted in slight increases in demand for cage-free and organic eggs, and a slight decrease in demand for conventional eggs, but did not cause a decrease in demand for eggs in general.”

Regarding the meat study: “While this is consistent with the hypothesis that media coverage of animal welfare causes a decrease in demand for meat, their evidence is not sufficient to substantially increase our confidence in this hypothesis.”—Models of Media Influence on Demand for Animal Products

The following document contains a template for evaluating virtually any intervention designed to help animals. We use this template to guide our intervention research and make it available for others to use in comparing interventions on their own. Observations about corporate outreach are outlined in the template below, as well as in the Evaluation and Error-tracking subsections for each Area.

There are many ways for an animal activism organization to pursue its goals, and choosing from these options requires understanding a number of quantitative and qualitative factors. Animal Charity Evaluators (ACE) finds it highly useful to standardize the methods by which proposed interventions are evaluated. This is why we use the following evaluation criteria. The criteria are divided into five areas of focus. For each criterion, we propose a method for summarizing everything that can be known about a given intervention. We hope that this will allow us to compare various campaigns across multiple dimensions and thereby maximize the reliability of our recommendations.

At the moment, there are many unknowns regarding the science, economics, and sociology surrounding animal activism interventions. Much of what is now known about how campaigns translate into reductions in animal suffering comes from small samples, anecdotes, or estimation. For this reason, this evaluation tries to provide as much information as possible, while still recognizing the limits of this young field of research. We do this by providing room for significant qualitative evaluation, while also attempting to give rough quantitative summaries of our work.

The evaluation is organized into five areas of focus. (I) type of intervention, (II) certainty of success, (III) barriers to entry, (IV) expected indirect effects, and (V) expected direct effects and overall efficiency analysis. Areas II through III might be of particular interest to organizations deciding what sort of intervention to pursue. Areas IV and V estimate the expected results from staging the intervention, both direct and indirect, and conclude with a calculation sheet that attempts a cost-benefit analysis.

Theoretically, this final cost-benefit consideration, which estimates the expected return on an organization’s investment, is the only important piece of this evaluation template. However, it is important to supplement this with additional reasoning regarding the likely effectiveness of interventions because we expect these calculations to include significant amounts of error and uncertainty. Such additions are especially valuable where they use different evidence than that which is primarily responsible for our cost-effectiveness estimate, since when only weak evidence is available, many independent lines of evidence can be analyzed, and if all agree, a strong conclusion can still be made. Additionally, given the diversity of organizations, with different resources, goals, and tolerance for risk, we thought it would be useful to give a more holistic picture of what each of our examined interventions entails, while still providing our best guess as to how efficient each one is at reducing animal suffering. As an illustration of the importance of including these auxiliary measures, an intervention with high barriers to entry (e.g. very large startup costs) might not be possible for some organizations, while an intervention with high uncertainties of success (i.e. higher risks) is likely to also require more estimations in our evaluations, and thus be prone to more error in our calculations.

Each area of interest includes an evaluation section with specific questions as well as areas for free-response. Additional forms allowing for summary of the evaluation overall are provided at the end of the document. We suggest using the provided forms during the evaluation process to ensure that all areas have been addressed.

This evaluation process is not without subjectivity. It also requires imperfect estimates of quantitative values. We hope that by publishing the materials used in conducting evaluations, these subjective judgements and estimates will be transparent to our audience. In the case of specific interventions, difficulties in evaluation may arise from:

limited data regarding certain aspects of past campaigns,

limited information about the connection between the immediate goals of the campaign and help for animals (e.g. neuroscientific or economic questions that have not been answered),

Even when it is imperfect, we believe that formal analysis is often a useful supplement to decision-making. We strive to make clear where there are gaps in our knowledge so that our conclusions can be integrated appropriately with other sources of information.

Summary of the Evaluation

investigation of an animal agriculture facility, sharing the information with the public and pressuring the company to make changes

Target Demographics

Target Animal(s)

Size of Campaign

Time Delay between Intervention and Results

consumers, corporations

varies, typically land animals

varies, usually several people working full- or part-time

immediately after release, but some results could occur years later

General Summary of Area I (Type).

Video investigations and rescues can take many forms. For instance, advocates performing open investigations may enter a farm or other facility with the clear intent of taking video to release to the public, generally at night in order to avoid being asked to leave before they are finished. Open rescues are usually performed similarly, but with advocates finding and rescuing some particularly distressed animals while they are at the facility. In undercover investigations, advocates disguise their intent at the farm or facility, either claiming to take video for an unrelated purpose or using hidden cameras and disguising the fact that they are filming at all.

We discuss any investigations and rescues where producing video footage and still images to distribute to the public is considered an important part of the project. Some investigations and rescues do not produce such outcomes; these are not addressed in our report. The types of investigations and rescues which are most common vary based on the laws and corporate cultures of individual countries. In general investigations seem to be more common than rescues. Undercover investigations in which investigators working on farms disguise the fact that they are filming at all are very common in the US, while in Europe less resource-intensive investigations are often possible.

Investigations can have negative consequences for farmers and farm-workers (who may be prosecuted for cruelty if a crime has occurred or face economic penalties due to bad publicity) and for corporations linked publicly to the farms investigated (which may also face economic penalties due to publicity). However, they are often considered to be in the (human) public interest, as they can reveal conditions which may be illegal or indicate problems with food safety.

Most organizations that are well-known for their investigation programs invest considerable effort in preparation for investigations and rescues and in carrying them out in the best and most professional way possible. Some investigations and rescues are carried out by individuals or organizations with less expertise; the people we spoke with were divided on how much expertise and preparation mattered.

Videos and still photos from investigations and rescues are extremely useful to organizations besides those who conducted the investigations, and provide a basis for much of the educational work that advocates do. Organizations carrying out investigations also work with other groups to ensure that they investigate any topics of particularly immediate interest, for instance releasing investigations relevant to ongoing corporate or legal campaigns that they are pursuing together with other organizations.

Code

Question
an (R) in the Score column indicates that a score of 1 would be generally considered preferable to a score of 7

Score
(1-7)

II.1.a

How many times has a similar intervention been attempted before?

7

Has a similar intervention ever been attempted in the same context as the one being evaluated?

6

II.1.b

How widely have previous campaigns varied in total results?

2

How widely have previous campaigns varied in efficiency?

2

How well have previous predictions of results matched actual results?

5

How well have previous predictions of costs matched actual costs?

5

II.2

How many different avenues of success does the campaign have available?

4

How many intermediate positive outcomes does the campaign have?

4

II.3.a

How well understood are the neurological abilities of the primary animal group targeted by the campaign?

5

How well understood are the neurological abilities of any other animal groups targeted by the campaign?

5

II.3.b

How well understood are individual and social group responses to this type of campaign?

3

How well understood are economic factors affecting the outcome of this campaign?

4

II.3.c

How well understood is the primary ecological impact of this intervention?

1

How well understood are any other ecological impacts?

1

II.4.a

How many known uncertainties affect the outcome of the campaign?

3(R)

How strongly do known uncertainties affect the outcome of the campaign?

5(R)

II.4.b

How many steps separate the actions of the campaign from a change in animal welfare?

6(R)

How many different actors are involved in the chain of events between the success of the campaign and the change in welfare?

6(R)

General Summary of Area II (Certainty of Success)

Undercover investigations of animal agriculture offer a relatively high certainty of success for the specific goal of reaching large numbers of people with information, although this varies significantly between campaigns and organizations. Investigations often lead to corporate policy change. Both these outcomes are directly observable.

We have more uncertainty about the outcome of consumer dietary change from being exposed to this information. Anecdotal evidence, such as self-reported interest in vegetarianism and veganism, is common, but there is little evidence for how many people make these changes for any given campaign. There are a small number of studies investigating the effect of animal welfare media coverage on demand for animal products, but their findings were very inconclusive.

As detailed in that report, the first study looked at the association of demand for eggs with press coverage of a farmed animal welfare ballot measure. Their model suggested that the proportion of eggs sold that were organic and cage-free increased while the proportion of conventional eggs decreased. The model did not output the effect on total demand, but total demand did rise after the ballot measure was passed, so the evidence might not even indicate a reduction in total egg production.

The second study looked at the association of meat demand with press coverage over a longer time scale across the US. This study did find a negative association (when there was more press coverage, there was lower demand), but this is only very weak evidence of a causal relationship. These studies might also not represent the true effects due to small sample size, issues with the data, or other reasons.

Undercover investigations might also cause indirect impact by increasing public opposition to animal agriculture or certain features of it. This could facilitate the success of future animal advocacy tactics such as corporate outreach or grassroots outreach. We have substantial uncertainty regarding the presence and magnitude of these effects.

Code

Question
an (R) in the Score column indicates that a score of 1 would be generally considered preferable to a score of 7

Score
(1-7)

III.1

How many specialized skill sets are required?

4(R)

How much expertise is required in the most demanding area?

6(R)

III.2.a

How demanding is the typical workload associated with the campaign?

7(R)

III.2.b

How many hours per week does the typical staff member work on the campaign when it is making its minimum demands?

3(R)

How many hours per week does the typical staff member work on the campaign when it is making its maximum demands?

6(R)

III.2.c

How many staff members are involved with the campaign when it is ongoing but at the time when fewest people are working on it?

2(R)

How many staff members are involved with the campaign at the time when the most people are working on it?

5(R)

III.3

How many rare or difficult-to-obtain materials does the campaign require?

2(R)

How severe is the difficulty of obtaining the hardest-to-obtain material needed for the campaign?

2(R)

General Summary of Area III (Barriers to Entry)

Undercover investigations that involve taking a job at an animal agriculture facility involve a substantial amount of resources. Resources can include hiring and training an investigator to do this work, having the proper recording equipment, seeking legal advice regarding the investigation, having the investigator undergo the emotional and physical strain of their work, having staff comb through the footage, and having the ability to share the footage with media outlets in a professional and effective manner.

Open investigations and rescues, or other similar activities that don’t involve employment at the facility, can probably be done with more limited resources. However, the basic requirements of a capable investigator (or several), recording equipment, and careful planning still apply. These costs can be reduced if the action is taken as part of an existing organization or network.

Several of these requirements seem to reduce as the organization conducting the investigations becomes more experienced. For example, recording equipment can be used for multiple investigations. Importantly, once an organization has established itself, getting attention from the press likely becomes much easier.

The intensity of work on a particular investigation can vary widely. The investigator might have to visit several facilities to find one where they can film. They might need to work in a position that doesn’t involve direct contact with the animals for a lengthy period of time before moving into a position where they can document cruelty.

Code

Question
an (R) in the Score column indicates that a score of 1 would be generally considered preferable to a score of 7

Score
(1-7)

IV.1.a

How large a chain effect does the intervention likely have?

6

IV.1.b

How likely is the campaign to inspire other victories?

4

IV.1.c

How significant a change in public mindset is this campaign likely to cause?

3

IV.2.a

How much alienation is this campaign likely to cause?

2(R)

IV.2.b

How high is the risk that the campaign may be more harmful than beneficial?

3(R)

General Summary of Area IV (Expected Indirect Effects)

Undercover investigations probably have some indirect effects through inspiring other organizations to conduct similar campaigns and providing an environment in which it is easier for future campaigns to gain more attention, although there are concerns about saturation at a certain number of investigations in a specific country or region.

We think the indirect effects are fairly large compared to most other intervention, mostly because of the press coverage they have received and how frequently they are referenced by the general public when farmed animals issues are brought up. The coverage of “ag gag” laws, for example, seems to have helped the general public appreciate the importance of helping farmed animals.

Final Total: the proposed intervention has a calculated efficiency of (_______), for a campaign of 20 weeks , with results being measured in years of farmed captivity (or equivalent suffering) averted.

Pessimistic

Realistic

Optimistic

0.15 lives/dollar

210 lives/dollar

820 lives/dollar

Were the optional modifiers used in calculating this total?

No

Final Determination (Overall Summary and Recommendations)

Undercover investigations are costly campaigns involving a large number of resources. They have huge potential to grow the animal advocacy movement and support other campaigns like corporate outreach and consumer dietary change. These effects are difficult to measure, especially in trying to attribute them to any specific campaigns, but in aggregate, it appears that undercover investigations have had a tremendous effect.

We recommend undercover investigations in most instances when done by experienced organizations, although we are concerned about saturation in the U.S. that could reduce the effectiveness of investigations beyond the current level, and the potential for investigations that are conducted poorly to reduce the overall reputation of animal advocates. We think that more research is needed on what makes some investigations more impactful than others, but tentatively, we consider thoughtful messaging, such as emphasizing that abuse is rampant throughout the industry and highlighting the stories of individual animals, as important. Additionally, undercover investigations seem particularly effective when coupled with other advocacy strategies, such as protests and corporate outreach, especially if they provide public pressure for a specific campaign, such eliminating battery cages.

Area I: Type

There are a number of ways to classify a proposed intervention. These variables will not be expounded upon in detail, but will be included so as to more accurately describe nuanced differences between various campaign types. If multiple campaigns are being analyzed to contribute to an understanding of the general efficacy of an intervention, some classifications may need to be applied to each campaign separately (see for example I.5 Size of Campaign.)

I.1 Activism Philosophy

A campaign might take an “incremental steps” approach to change, or follow a hardline all-or-nothing mentality. It might work to reform existing institutions or to oppose them completely. The following approaches are helpful to consider:

Animal welfare approach / animal protectionism: attempting to improve the living conditions of animals without necessarily opposing a priori their use by humans for food/labor/etc.

Animal rights approach: attempting to enforce a set of rights guaranteeing animals certain inalienable protections from harm. This could include such things as a right to be free from involuntary confinement regardless of quality of life.

Abolitionist approach: a sub-category of the animal rights approach. It seeks to achieve its goal of ending all uses of animals as human property by an all-or-nothing approach. It does not accept incremental change if such change allows for the continued use of animals as human property.

Antispeciesism approach: treating certain abuses of animals as discrimination on the basis of species membership. This is not necessarily mutually exclusive with the previous two approaches.

I.2 Activism Approach

An intervention might work through direct action, legislative or political lobbying, research and development, or other means.

I.3 Target Demographic

The target demographic is the intended audience for a campaign. A campaign may target multiple demographics, and for some interventions, different campaigns may target different demographics.

I.4 Target Animal(s)

Target animals are the intended beneficiaries of a campaign.

I.5 Size of Campaign

The size of a campaign can be measured in terms of its duration, the amount of money available to sustain it, the number of people involved, and the size of the sponsoring group.

I.6 Time Delay between Intervention and Results

Time delay is a measure of how long will it take for a campaign to effect meaningful change.

I.7 Increase/Decrease in Human Suffering

Humans are animals too, and the effects of a campaign upon human welfare should not be completely ignored.

I.8 Characteristics of Organization Conducting Intervention

The success of a campaign may depend on aspects of the organization conducting the campaign including its age and size, track record with other interventions, quality of staff, and degree of commitment to the intervention. This section is particularly important for determining the applicability of results to campaigns conducted by organizations other than those whose work is considered in this evaluation.

I.9 Appeal to other Organizations

A campaign may appeal to other organizations, funders, or individuals, bringing in support that could not be planned for from its launch. For past campaigns, to what extent has this occurred, and with what organizations? For possible campaigns, what other organizations may lend support, and what circumstances would make this most likely?

I.10 Evaluation

Activism Philosophy

Activism Approach

Undercover (and other) investigations often take a welfare-oriented approach, pointing out specific cruelties experienced by farmed animals. Rescues can be viewed as a welfare intervention but are more often designed to have an animal rights/antispeciesist message.

Video from investigations and rescues is often used for primarily educational purposes; it is distributed to the public to demonstrate what animals experience on farms. Some organizations also pursue legal action or corporate policy change based on the results of their investigations.

Target Demographics

Target Animal(s)

Size of Campaign

Time Delay between Intervention and Results

broadly targeted to consumers in a region/country. Sometimes a specific demographic such as policy makers is particularly targeted by the choice of media to take the exclusive.

for investigations, most often larger animals like cows or pigs; for rescues, usually small animals like poultry

Ranges from effort of a few days to many months working on the project; investigations and rescues often fit into larger campaigns lasting months to years.

Time delay between intervention and people seeing footage can be very short (days or weeks); to corporate policy change, weeks to years; to legal action, days to weeks

General Summary of Area I (Type). Please also address, if applicable, relevant details about how the intervention is likely to affect human interests; the capability of the organization planning the intervention; and the intervention’s appeal to other organizations.

Video investigations and rescues can take many forms. For instance, advocates performing open investigations may enter a farm or other facility with the clear intent of taking video to release to the public, generally at night in order to avoid being asked to leave before they are finished. Open rescues are usually performed similarly, but with advocates finding and rescuing some particularly distressed animals while they are at the facility. In undercover investigations, advocates disguise their intent at the farm or facility, either claiming to take video for an unrelated purpose or using hidden cameras and disguising the fact that they are filming at all.

We discuss any investigations and rescues where producing video footage and still images to distribute to the public is considered an important part of the project. Some investigations and rescues do not produce such outcomes; these are not addressed in our report. The types of investigations and rescues which are most common vary based on the laws and corporate cultures of individual countries. In general investigations seem to be more common than rescues. Undercover investigations in which investigators working on farms disguise the fact that they are filming at all are very common in the US, while in Europe less resource-intensive investigations are often possible.

Investigations can have negative consequences for farmers and farm-workers (who may be prosecuted for cruelty if a crime has occurred or face economic penalties due to bad publicity) and for corporations linked publicly to the farms investigated (which may also face economic penalties due to publicity). However, they are often considered to be in the (human) public interest, as they can reveal conditions which may be illegal or indicate problems with food safety.

Most organizations that are well-known for their investigation programs invest considerable effort in preparation for investigations and rescues and in carrying them out in the best and most professional way possible. Some investigations and rescues are carried out by individuals or organizations with less expertise; the people we spoke with were divided on how much expertise and preparation mattered.

Videos and still photos from investigations and rescues are extremely useful to organizations besides those who conducted the investigations, and provide a basis for much of the educational work that advocates do. Organizations carrying out investigations also work with other groups to ensure that they investigate any topics of particularly immediate interest, for instance releasing investigations relevant to ongoing corporate or legal campaigns that they are pursuing together with other organizations.

Area II: Certainty of Success

Campaigns with identical efficiency ratings can still be differentiated. One way to do this is to focus on the riskiness of an intervention. If efficiency is held constant, riskier campaigns offer a relatively low chance of achieving a relatively high level result. If our estimates of expected value were perfect, we might not care about certainty of success except as it affects organizations’ willingness to attempt a campaign. However, we will not be able to provide perfect estimates of expected value, because in the real world we can never have perfect information. Our estimates of expected value are particularly likely to deviate from reality in the case of risky campaigns, since they succeed only some of the time, and, based on a finite and often small number of attempts, we cannot know the actual success rate with great precision.

II.1 Precedents

The success of future campaigns can be estimated in part by the success of past similar campaigns.

II.1.a Existence of previous attempts

To base predictions on past performances, it must first be established whether or not the proposed intervention has ever been attempted before. If so, it will be useful to know how many times it (or something similar) has been attempted, and whether those attempt(s) were staged in the same setting and context as the intervention currently being considered. The setting and context include such considerations as target demographic, geographical location of campaign, etc. It is generally assumed that unprecedented campaigns are, all things being equal, riskier than campaigns with solid precedents.

II.1.b Variation in efficiency of previous attempts

If a similar intervention has previously been staged, it is important to understand the variation in both costs and results. Campaigns types with a history of unpredictable costs and outcomes are riskier than those with a less chaotic performance record. Wild fluctuations in run costs or in successful outcomes achieved will translate directly to wildly fluctuating measures of efficiency.

II.2 Flexibility of Goals

Campaigns that have lofty, all-or-nothing goals are inherently riskier than campaigns with continuous spectrums of potential positive outcomes. Campaigns that do not have fallback victories will be riskier than campaigns that do. For example, a low-visibility campaign to convince a corporation to stop supporting a certain type of animal testing may fail to produce any positive result if it is not successful. It is thus important to find out if there is only one goal set for the campaign, or if there are a variety of possible outcomes that will satisfy the purpose of the campaign. Campaigns that cast wider nets are less risky than ones that focus on narrow areas of improvement. For example, a gestation crate awareness campaign may have multiple potential goals, including getting its audience to adopt vegan diets, getting its audience to purchase pork from more humane sources, etc.

II.3 Scientific Certainty

There are a variety of scientific disciplines that study issues relevant to the success of animal activist interventions. The extent to which scientists have succeeded at understanding factors relevant to a particular intervention affects our certainty that the intervention will have a positive effect.

II.3.a Nonhuman neuroscience

There is not always clear scientific consensus regarding the neural complexity of nonhuman animals. A greater understanding of an animal’s neural networks allows us to be more certain that it is affected by allegedly harmful practices. For example, a campaign focusing on fishing may have to temper its scientific certainty more than a campaign focusing on pigs (given what is known about the latter’s neuroanatomy, and what is not known about the former’s). In making this determination, it may be necessary to read into the neuroscientific and/or psychological literature on the target animal population in question. We expect this to be a particularly controversial area of research, which will likely evolve along with scientific understanding of nonhuman animal consciousness.

II.3.b Sociology, psychology, and economics

Many campaigns rely upon assumptions about how humans will respond to an intervention, how societies will adapt to proposed change, and/or how economic forces will react to a campaign. It is thus necessary to uncover any ambiguities in the relevant literatures supporting the social science assumptions of an intervention. For example, the reliability of data on recidivism rates amongst new vegans will be important in estimating the certainty of success of a diet-focused campaign.

II.3.c Ecology

The expected effects of an intervention may rest on ecologically-based predictions. These interventions also must be rated based on reliable ecological data. For instance, it may be important to understand how factory farms impact their surrounding environment before being able to reliably quantify the effects of an anti-factory farming campaign.

II.4 Other Uncertainties

There may be other miscellaneous risk factors not yet addressed.

II.4.a Known unknowns regarding an intervention

Some risks are unpredictable, but others may be anticipated with proper planning. Before beginning an intervention, it is advisable to go through all components of a plan of action in order to determine what might go wrong, however unlikely. This is purposefully vague, as this method can be thought of as a “catch-all” for types of uncertainties thus far not addressed. For example, a certain legislative campaign may have to take into consideration the possibility of a cooperating political partner losing a reelection bid. Another campaign may have to take into consideration unfamiliarities in the culture of their target demographic, or a lack of certainty that all members of the campaign team can stay on for the duration of the campaign.

II.4.b Distance between intervention and improvement in animal welfare

The distance between a campaign and the animals it intends to benefit can be defined as the degree of separation or number of steps between an intervention’s activities and the end result of actually helping animals. For example, a campaign to care for rescue animals has a direct impact on animal welfare, while a campaign to train new animal activists would expect to achieve far-removed, indirect results (training → new activists → new campaigns → improvement in animal welfare). Similarly, a campaign to pass a new animal welfare law would also be further removed than caring for animals directly. A greater distance between intervention and result leads to a greater uncertainty of success.

II.5 Evaluation and Error tracking for “Certainty of Success”

Evaluation for II.1.a

How many times has a similar intervention been attempted before?

1

2

3

4

5

6

7

1= No past attempts
7=Multiple organizations have started this type of campaign each year for at least the past decade.

Notes: Undercover investigations are very common in the US, and multiple organizations have been attempting them yearly (often several times per year) since the early 2000s. Other types of video investigations and rescues are less common or performed by fewer groups, especially within a particular country. (For instance, open rescues have been rare in the US in the recent past, although they have been more common in parts of Europe and perhaps Australia.) While some aspects of an investigation or rescue are similar regardless of where it is performed, legal risks and challenges imposed by farmers can vary significantly between countries.

Has a similar intervention ever been attempted in the same context as the one being evaluated?

1

2

3

4

5

6

7

1= Few similarities, and no exact parallels, exist between past campaigns and one being evaluated, in terms of target demographic, geographic location, organizational similarity, etc. … 7=Multiple campaigns have been previously attempted that have identical or near identical contexts as the one being evaluated, in terms of the factors listed above.

Notes: Most campaigns target very similar demographics (roughly, a general news audience, or a slightly more educated news audience) and are performed in countries where other similar campaigns have previously taken place, in fact often by the same organization within the past year. However, organizations try to avoid running consecutive campaigns that are similar in terms of species addressed and geographic location, because it is easier to get media coverage for investigations on new topics. Some of the most successful campaigns are the most novel in this way; campaigns which portray activities the public did not already know about are considered especially newsworthy. This also means that a campaign which is extremely similar to a recent previous campaign cannot be expected to have the same level of success, because the previous campaign changes the context for the new campaign in a meaningful way.

Evaluation for II.1.b

How widely have previous campaigns varied in total results?

1

2

3

4

5

6

7

1= Very wide variation; some campaigns have achieved remarkable results, while others have been completely unsuccessful or counterproductive
7=Very little variation; it is difficult to detect any meaningful difference in results

Notes: Campaigns vary widely in results, in part because of their dependence on cooperation from outside parties for the best possible outcomes. Some campaigns receive considerable national or international media coverage, while others are disseminated primarily via the conducting organization’s social media presence. This can result in significant differences in the number of people who view an investigation or rescue, and also in how seriously it is taken by corporate policy makers. Additionally, some investigations result in major policy changes by large corporations while others are not directly linked to this type of change (or only to smaller changes of a similar type); this has to do with both the strategies and capabilities of the conducting organization and hard-to-predict characteristics of corporations targeted for change.

How widely have previous campaigns varied in efficiency?

1

2

3

4

5

6

7

1= Very wide variation; some campaigns have had large results with few resources, while others have been completely unsuccessful or counterproductive
7=Very little variation; all variation in results can be explained by variation in resources put in

Notes: Campaigns vary widely in efficiency in part due to differences in strategies for obtaining access to animals and video footage, and in part due to differential follow-up and differential cooperation by outside actors such as corporations and the media.

Comment on what is known about the reason(s) for the above mentioned variations between campaigns. Specifically, what variables were external and which were specific to a particular campaign/organization.

Organizations are able to control the subject of an investigation/rescue and to some extent the method they use to conduct the investigation. For instance, they can choose whether to conduct an investigation or a rescues, and whether to take footage in multiple locations for a combined release or to have one investigation correspond to one farm. There are also some characteristics that are under organizational control to some extent but are difficult to change for a given project: for instance, some organizations have departments devoted to corporate outreach and experienced personnel with media connections, while others don’t; these differences can affect how much coverage an investigation is likely to receive, or how likely it is to lead to corporate change. (Although organizations with fewer resources in these areas can still get some of them same effects by working together with other organizations or investing more time in securing media coverage.)

Some characteristics are less under organizations’ control. The fastest and cheapest way to obtain footage of farm conditions is to do open investigations, which involve trespassing on farms at night. Such investigations are safe even for established organizations in some countries, where even trespassing charges will be thrown out on the grounds that investigators were doing a public service, but can be risky in other areas which are less sympathetic to investigators. Undercover investigations in which activists film openly but under a pretext also have advantages, most notably the high quality of footage that can be obtained, but are difficult or impossible to conduct in areas where farmers are aware of the possibility of investigations.

Media and corporate responses are also clearly outside organizations’ control, although organizations which conduct many investigations or rescues can develop skills that help them identify investigations which are likely to be of strong interest to the media or corporations it is particularly valuable to target. These factors seem to be subject to considerable chance even for the most experienced organizations, however.

How well have previous predictions of results matched actual results?

1

2

3

4

5

6

7

1= Very poorly; advance predictions do not appear to correlate with campaigns’ eventual success
7=Very well; successful campaigns were in general widely expected to succeed and unsuccessful ones were clearly longshots

Notes: There is some evidence that organizations are able to predict which investigations and rescues will get good results in the media. There is much less evidence that they’re able to predict which investigations will get good results from corporations (especially because it is difficult to plan an investigation to affect a particular corporation).

How well have previous predictions of costs matched actual costs?

1

2

3

4

5

6

7

1= Very poorly; most campaigns have either raised funds they were unable to use or had unexpected expenses leading them to need more funding than projected
7=Very well; campaign costs have generally matched the planned budget closely

Notes: Organizations that perform many investigations seem to have a fairly good sense how much each will cost and to be able to control costs to some degree. However, there are still random factors, such as the length of time that investigators spend looking for work in the case of undercover investigations during which they work on the investigated farm. This variance can amount to a noticeable proportion of the budget of such investigations.

Evaluation for II.2

How many different avenues of success does the campaign have available?

1

2

3

4

5

6

7

1= The campaign has only one major goal or path towards success
7=The campaign has several distinct goals that could each be reached independently

Notes: Campaigns usually have 1-2 main goals, general public education and progress on some specific issue (such as a change in corporate policy, a shift away from eating a certain type of product, etc). Usually organizations have multiple media sources they can approach to cover the investigation or rescue, and they can also share the video through their own social media presences, which are becoming increasingly significant.

How many intermediate positive outcomes does the campaign have?

1

2

3

4

5

6

7

1= Partial success on any of the campaign’s goals has no clear benefits
7=Any step towards any of the campaign’s goals has proportional benefits

Notes: Spreading public awareness has benefit at any scale, although on the smallest scale, if an investigation only reaches an organization’s immediate followers, it is likely not to be doing nearly as much good as if it also reached a less-affiliated audience. Partial progress towards other goals, such as corporate or legal policy change, may be very different in character from full progress or even entirely futile.

Evaluation for II.3.a

How well understood are the neurological abilities of the primary group of animals targeted by the campaign?

1

2

3

4

5

6

7

1= Understood mostly on an intuitive level
7=Well understood behaviorally and on a neurological level

Notes: Most investigations target animals whose neurological abilities are well understood, since this makes their suffering more interesting and compelling to the media and a general audience. Rescues must also target animals which are easy to take care of, and this can mean smaller animals whose cognitive abilities are less well understood. (In particular, birds rather than mammals.)

How well understood are the neurological abilities of any other groups of animals targeted by the campaign?

1

2

3

4

5

6

7

1= Understood mostly on an intuitive level
7=Well understood behaviorally and on a neurological level

Notes: Neither rescues nor investigations typically target animals with cognitive abilities very different from those of chickens, pigs, or cows.

Comment on the choice of target group(s) of animals by the campaign. Address how well the goals of the campaign align with the ability of the recipient animals to meaningfully benefit from those goals, based on what is known about their neurobiology.

Most groups focus investigations on farmed animals who are clearly suffering and would benefit from improved conditions or a reduction in the number of animals being farmed. Indeed, because the success of video investigations and rescues depends on the need for improvement being clear to anyone who views a video, and people tend to be overly skeptical of animals’ ability to feel pain and to benefit from improved conditions, campaigns tend to focus on animals who are not just known to be able to benefit, but who are the most clearly known to be able to benefit, and on conditions which are the most clearly unacceptable.

Evaluation for II.3.b

How well understood are individual and social group responses to this type of campaign?

1

2

3

4

5

6

7

1= Understood mostly on an intuitive or theoretical level
7=Many rigorous studies show very similar responses

Notes: Some studies show that news coverage of the condition of animals tends to temporarily reduce demand for animal products (at least in the case of certain animals). However, the body of research is not large and deals only with fairly short term effects. It’s not clear to what extent video investigations and rescues produce long-term changes in thinking, or to what extent the promote welfare vs. rights frames of mind. The studies also do not deal with the ways that people interact socially around animal products, aside from purchasing decisions. There is some anecdotal evidence in this area, such as through the responses of executives or politicians to individual campaigns, the spread of petitions, etc.

How well understood are economic factors affecting the outcome of this campaign?

1

2

3

4

5

6

7

1= Understood mostly on an intuitive or theoretical level
7=Many rigorous studies show very similar responses

Notes: Some studies show that news coverage of the condition of animals tends to temporarily reduce demand for animal products (at least in the case of certain animals). However, the body of research is not large and deals only with fairly short term effects. Some organizations also have regularly engaged with corporations about their investigations, and therefore have a body of anecdotal evidence regarding possible corporate responses.

Evaluation for II.3.c

How well understood is the primary ecological impact of this intervention?

1

2

3

4

5

6

7

1= Understood mostly on an intuitive or theoretical level
7=Many rigorous studies show very similar responses

Notes: Video investigations and rescues have minimal direct ecological effects. Rescues result in a small number of animals living longer lives in caring homes, which might use resources in a way that affects the farmland/wild balance. Indirect effects might also result from demand for animal products being reduced as a result of media coverage (less farmland used in expectation) or changes in corporate or legal policy governing farming methods (possibly farmland being used less intensively, possibly more or less farmland being used or waste products being disposed of more carefully). None of these effects have been carefully studied.

How well understood are any other ecological impacts?

1

2

3

4

5

6

7

1= Understood mostly on an intuitive or theoretical level
7=Many rigorous studies show very similar responses

Notes: No possible ecological effects of video investigations and rescues have been carefully studied.

Comment on any perceived oversights or uncertain assumptions in the scientific, sociological, economical, and/or ecological rationale behind this campaign.

The most significant gaps in the rationale behind video investigations and rescues are in the evidence for their having the desired social and economic effects. The evidence for the social effects is particularly thin; because video investigations and rescues are meant to affect very large numbers of people through media coverage, but not expected to have an especially strong effect on most of the people they reach, it’s very hard to track their effects with certainty. As a result, organizations make decisions about video subjects, moods, and themes based mainly on anecdotal evidence or on analogies to other situations. For instance, they might choose to focus a video on an individual animal because people have been shown in other contexts to respond better to individual-focused narratives than narratives that portray suffering on a larger scale, but we don’t actually have evidence showing this applies specifically to the situations in which people encounter video from investigations and rescues. It’s also very difficult to determine what actions people really take as a result of seeing the videos, especially if they see them in mass media (where organizations are not able to track individual viewers at all). While organizations make credible claims that people change their diets, change their views, and become involved in activism as a result of seeing videos from investigations and rescues, it’s very uncertain how many people do each of these things.

There is more evidence for the economic effects of the videos, because anecdotal reports of companies changing their policies represent greater fractions of the total behavior of interest. There are also a small number of studies investigating economic effects of media coverage on overall demand, which found inconclusive results.

Finally, video investigations and rescues have the same scientific uncertainties with regard to the sentience of commonly farmed animals and the ecological effects of lessening animal farming (or slowing its growth) that most other interventions targeted broadly at farmed animals have. For instance, it’s possible that pigs really can’t feel pain, or that farmed animals generally have net-positive lives, though we think both these concerns are unlikely. It’s also possible that reducing the amount of animal farming will increase (or decrease) the number of wild animals living at any given time over the long run, with attendant concerns depending on whether these animals’ lives are worth living.

Evaluation for II.4.a

How many known uncertainties affect the outcome of the campaign?

1

2

3

4

5

6

7

1= None
7=Several different sources of uncertainty not addressed above

Notes: The primary other uncertainties affecting the outcome of a given video investigation/rescue are the reactions of various institutions to that campaign. These are partially (especially in the case of corporations) addressed above, but not fully. In particular, media response to an investigation or rescue often determines how many people see the video produced, and can’t be entirely predicted ahead of time. Also, the corporate response to an investigation or rescue isn’t entirely predictable ahead of time and could range from laying charges against the activists involved in producing the video (especially if they engaged in any illegal activity), to stating that their practices are not being accurately reflected by the video, to no response, to making changes in their welfare policies affecting thousands or millions of animals.

How strongly do known uncertainties affect the outcome of the campaign?

1

2

3

4

5

6

7

1= No known uncertainties affect this campaign
7=The success of the campaign hinges upon one or more uncertain factors not explicitly addressed by the campaign itself

Notes: Media response in particular carries significant weight in determining the impact of a campaign. However, even were no media outlet to be interested in reporting on a given campaign, most organizations at this point can achieve a moderately wide distribution of materials through social media.

What outside actors and external events are expected to be able to affect this campaign’s success?

Media – What media outlets want to cover the campaign/how they cover it.

Other campaigns by other groups – Similar campaigns released at similar times might make the campaign less interesting and newsworthy, or make the problems it exposes look more pervasive.

Corporations – Whether e.g. grocery stores pressure suppliers to change practices as a result of the video.

Law enforcement – response can affect how the campaign is covered in the media

What unanticipated problems have befallen past campaigns? How were they handled?

One common source of problems is obtaining access to film animals. Organizations have come up with many ways of obtaining access to farms, despite that farmers would generally prefer that investigations not take place. Investigators using undercover tactics, in particular, have modified the equipment and cover stories that they use as farmers implement more barriers to filming. Organizations also encourage investigators to attempt to gain access to many farms, even if a smaller number are ideal targets, to speed up the process of gaining access to some farm.

Sometimes organizations have found that the media is less interested in a particular campaign than expected. In general this seems to be solved for the particular campaign by releasing it through smaller media outlets or the organization’s own social media, leading to less progress on campaign goals, but still some progress. Organizations that encounter this problem often take steps to avoid it in future campaigns, for example, paying greater attention to spacing out campaigns focusing on a specific species or relevant to a particular area. These steps seem to be moderately effective.

Evaluation for II.4.b

List the chain of events connecting this intervention to a change in animal welfare, in as much detail as possible.

Advocates gain access to film and possibly rescue animals by:

breaking into farms, usually after dark.

posing as researchers or others with an interest in filming farm activities and contacting farmers to set up farm visits. This may involve complicated steps such as producing business cards, developing a cover story, providing references, etc.

seeking employment at farms. Typically an investigator applies to many farms in an industry and area and takes the first job available. Some jobs don’t actually involve contact with animals, in which case the investigator must seek a transfer or new job.

Advocates film animals on a farm or in another industrial agriculture setting (e.g. transport, a slaughterhouse). This may take a single night, repeated visits, or weeks of filming secretly during work hours.

Advocates may rescue a small number of the animals they see, especially if they are present on the farm without the farmer’s knowledge.

Advocates may repeat the above steps for more farms in the same industry, later combining the footage in one investigation video.

Advocates review the video footage.

If the footage depicts crimes being committed, advocates may share their findings with local law enforcement and wait to proceed with other steps until law enforcement has taken appropriate action.

In this case, law enforcement may charge individuals with animal cruelty, or in extreme cases require a facility to be closed.

If individuals are convicted of cruelty, or especially if a facility is closed, economic theory predicts that the supply curve for the product(s) in question will shift, leading to slightly higher prices and lower production.

Action by law enforcement may also lead to more media coverage (see below).

Advocates may also approach corporations which buy products from the investigated facility.

In this case, they must generally research the connections between the facility and other companies in the supply chain, since these connections are usually proprietary information.

Advocates use a variety of methods to influence corporate decisions and through them animal welfare. See our evaluation of corporate outreach.

If a rescue was involved, advocates provide the rescued animal(s) with veterinary care and find them a new home where they will be cared for in future.

Advocates edit the video, photos, etc. produced by the investigation for the media and the public.

Advocates contact the media and arrange to have the investigation/rescue released exclusively by an outlet (thus securing coverage by an outlet which would not otherwise cover the investigation).

Advocates arrange for other coverage, by arranging a press conference, follow-up interviews in other media, and by releasing the investigation on their own website and through social media.

People see coverage of the investigation in the media and on social media. They may spread it further through social media and discussion.

People acquire an unfavorable impression of animal agriculture from the investigation.

People change their animal product consumption, eating fewer animal products and/or purchasing them from higher welfare sources. Economic theory predicts that the demand curve for the (lower-welfare) products in question will shift, leading to lower prices and lower production.

Images, stories, and video from the investigation or rescue may be used in educational materials after the campaign, making them more effective than if the next-best materials were used instead.

How many steps does separate the actions of the campaign from a change in animal welfare?

1

2

3

4

5

6

7

1= None; the campaign helps animals directly
7=Very many; the chain is so long that it is difficult to think about in detail

Notes: The chain is quite long for the most common method of influence (public education) and rather uncertain past the point of media/social media exposure. If corporate outreach or legal action is also involved, the chain is somewhat shorter in that area.

For rescues, a few animals experience a change in welfare as a very direct result of the campaign.

How many different actors are involved in the chain of events between the success of the campaign and the change in welfare?

1

2

3

4

5

6

7

1= None; the campaign helps animals directly.
7=Very many; the chain is so long or involves such complicated events that it is difficult to identify all actors or groups involved

Notes: Many different actors are involved, but generally all can be identified.

Summary

General Summary of Area II (Certainty of Success). Summary may include relevant comments about the following topics: what has been learned from previous campaigns; how narrow or flexible are the goals of the campaign; how clearly is the necessary scientific, sociological, economic, and ecological science upon which the campaign’s success relies understood; what is the (metaphorical) distance between the campaign and the animals it is attempting to help; and what other uncertainties should be noted.

Undercover investigations of animal agriculture offer a relatively high certainty of success for the specific goal of reaching large numbers of people with information, although this varies significantly between campaigns and organizations. Investigations often lead to corporate policy change. Both these outcomes are directly observable.

We have more uncertainty about the outcome of consumer dietary change from being exposed to this information. Anecdotal evidence, such as self-reported interest in vegetarianism and veganism, is common, but there is little evidence for how many people make these changes for any given campaign. There are a small number of studies investigating the effect of animal welfare media coverage on demand for animal products, but their findings were very inconclusive.

As detailed in that report, the first study looked at the association of demand for eggs with press coverage of a farmed animal welfare ballot measure. Their model suggested that the proportion of eggs sold that were organic and cage-free increased while the proportion of conventional eggs decreased. The model did not output the effect on total demand, but total demand did rise after the ballot measure was passed, so the evidence might not even indicate a reduction in total egg production.

The second study looked at the association of meat demand with press coverage over a longer time scale across the US. This study did find a negative association (when there was more press coverage, there was lower demand), but this is only very weak evidence of a causal relationship. These studies might also not represent the true effects due to small sample size, issues with the data, or other reasons.

Undercover investigations might also cause indirect impact by increasing public opposition to animal agriculture or certain features of it. This could facilitate the success of future animal advocacy tactics such as corporate outreach or grassroots outreach. We have substantial uncertainty regarding the presence and magnitude of these effects.

Error Tracking

How rigorously documented were the reports from past similar campaigns? How reliable is the information that defines the precedents for the campaign?

Information on the media and corporate policy outcomes of previous campaigns is fairly reliable in the aggregate. We have spoken to advocates who have significant experience conducting undercover investigations, and we can see the corporate policy announcements and media coverage ourselves. The information on the indirect outcomes of undercover investigations is much less reliable, and it would be very challenging for people running these campaigns to document these outcomes for each campaign.

What ambiguity exists in the scientific consensus regarding the mental abilities of relevant animals, the sociological and economic effects necessary for the campaign to make a difference, and the campaign’s ecological effects?

Undercover investigations target either all farmed animals or cases where the ambiguity around animals’ ability to benefit from changes to their situations is deliberately chosen to be small. While there is substantial ambiguity in the scientific consensus about animal consciousness and suffering overall, undercover investigations are not more subject to this ambiguity than other interventions.

What assumptions were made regarding “other considerations?” What factors may not have been considered?

The above assessment applies primarily to undercover investigations conducted by groups with existing investigation programs, or by groups with similar agendas and abilities. Because it is taken from the experience of relatively few groups, it is possible that some other groups would have very different experiences with otherwise similar campaign attempts, because of other group characteristics.

Area III: Barriers to Entry

There are a myriad of factors to consider when evaluating how difficult an intervention will be to stage. This area of the evaluation examines the complexity of human skill required, the degree of work intensity needed, and the difficulty involved in obtaining the required material resources. This is not meant to duplicate the valuation mechanism laid out in Area V (Efficiency), but rather to provide a separate metric to measure the challenges inherent in putting on a campaign, with no reference to a conversion to fiscal units. This provides part of a cross-check for Area V, since the higher the barriers to entry for a campaign, the better its probable results must be to maintain a constant level of efficiency. In addition to providing a check on the efficiency as calculated below, knowing the barriers to entry for an intervention can help identify which types of group can likely conduct it successfully.

III.1 Skill Required

It is important to know what sort of expertise is required from a campaign’s staff. The more specific knowledge and/or special skills required, the more difficult a campaign will be. For example, a campaign to challenge an anti-whistleblower law will need significant legal expertise and political skill.

III.2 Work Required

The calculation of work required can be defined as a calculation of the intensity of effort required from an intervention’s staff members.

III.2.a Intensity of the work

Intensity is an attempt to quantify how exhausting, physically taxing, and mentally draining campaign-related work is. A more demanding workload indicates a more difficult campaign plan. For example, a campaign to engage strangers on a city street and talk to them about veganism may require exhausting days with lots of mentally taxing social interaction.

III.2.b Hours per week

The number of hours of work per week demanded from staff members compounds the effect of the intensity of their work per unit time. It will be useful to know how many hours per week are required from each type of staff member both at a minimum and at a maximum. A campaign may require a greater commitment around major events.

III.2.c Number of people needed

The number of staff members required for a campaign can be a limiting factor. Like the number of work hours required per week, this value may fluctuate over time.

III.3 Resources Required

It is further useful to examine the complexities involved in obtaining the necessary resources for an intervention. The procurement of materials for an intervention may require difficult-to-obtain permissions, such as copyright waivers, or an extensive search for uncommon objects. For instance, a campaign to develop a new meat substitute that requires a rare chemical has inherent difficulties associated with resource obtainment.

III.4 Evaluation and Error tracking for “Barriers to Entry”

Evaluation for III.1

How many specialized skill sets are required?

1

2

3

4

5

6

7

1= None
7=Many unrelated and specialized skill sets

Notes: The campaign requires at minimum contact with an expert on the legal issues involved, a good video editor, and someone with the patience and professionalism to contact the media. Several other skilled positions are useful but optional.

What special skills are required in conducting this campaign?

Required:

Investigators able to remain calm in upsetting and potentially dangerous situations.

Legal expertise in areas related to the investigation, or legal advice from someone with such expertise.

Video editing skills, to produce a short and compelling video from possibly very large amounts of footage.

Communication skills for contacting and interacting with the media.

Nice but not necessarily required:

Investigators able to hide their emotional reactions to animal suffering/play the role of someone expected to be in the facility.

Skills with undercover camera and video equipment.

Still photography skills.

Video recording skills, for making original footage compelling.

Writing skills, for adding a narrative to final video.

Voice acting skills, for recording the narrative for the final video.

Research skills, for connecting the facility to major corporations.

Communication and negotiation skills, for interacting with corporate executives.

Web design and coding skills, for building a website to host the video.

Notes: Legal advice/expertise should be provided by a lawyer, ideally one with experience in the type of investigation or rescue being performed or situations with similar legal implications.

Comment on the expected ease with which qualified staff members will likely be recruited. This may involved such factors as the job market for a certain profession, the average interest in animal activism from people with certain qualifications, etc.

The many positions involved (or potentially involved) in video investigations and rescues require a variety of different characteristics from advocates. In most cases, they require the involvement of several different people.

All positions but the lawyer and investigator roles seem generally appropriate to fill with smart, competent animal advocates who may not initially have the required skills but can develop them. Video investigations and rescues benefit from having competent and versatile people in these positions, who certainly could be useful on other campaigns, but organizations don’t seem to have particular trouble recruiting them.

Investigators can also be any competent and professional animal advocates, in the case of open rescues and investigations and perhaps some undercover ones. But undercover investigations in the US must use investigators who are not identifiable as animal activists, because farms commonly run background checks on people applying to work with animals, in part to prevent investigations. As increasing amounts of information on people’s life histories and political views are available online, this means many animal advocates who might want to serve as investigators are unable to do so. However, organizations still seem to be successfully recruiting an adequate or above-adequate number of investigators from among people who have recently developed an interest in the treatment of farmed animals and don’t have an online record tying them to animal advocacy.

Lawyers must have the most specialized training, so in some ways it seems like they would be hardest to recruit. But in fact many lawyers enter the profession in order to do good, including good for animals, and jobs with respected animal organizations are highly sought-after, at least in the US. Especially in the current job market, recruiting young lawyers is fairly easy. Lawyers with specialized experience are more difficult to find, but the ALDF provides legal advice to animal advocates, and their staff is especially knowledgeable and qualified. Getting the ideal legal advice may be difficult (and could be important, especially in the US), but getting a reasonable quality of legal advice seems fairly easy for an organization willing to explore its options.

Evaluation for III.2.a

How demanding is the typical workload associated with the campaign?

1

2

3

4

5

6

7

1= Light; working on the campaign involves little stress and is likely pleasant
7=Heavy; working on the campaign involves significant physical or mental labor at most times

Notes: Working on the campaign involves significant labor, especially emotional labor in confronting and working near animal cruelty, both at the farm or facility and during video editing, two of the longest phases of a normal campaign.

What challenges are involved with the daily work of the campaign?

For investigators, challenges involve bearing witness to animal suffering, making quick decisions about how to cope with challenging situations (the risk of discovery, which animals to rescue (if any)), possible hard physical labor, and possible interaction in which they cannot reveal their genuine purpose.

Others involved in the campaign face more intellectual challenges, such as how to work within the local laws and how best to edit materials and compose statements presenting the campaign to law enforcement, corporations, the media, and the public. Advocates in these positions may also face emotional challenges in facing animal cruelty, especially if they are required to review significant quantities of video footage.

Evaluation for III.2.b

How many hours per week does the typical staff member work on the campaign when it is making its minimum demands?

1

2

3

4

5

6

7

1= 1 or fewer
7=80 or more

Notes: Because campaigns vary so much, there’s no estimate for even a single role (e.g. investigator) that would be appropriate for all campaign types. In most cases, not all participants will be actively working on the campaign at all times, but some staff member would be doing several hours of work per week (though less than 40) even when the campaign is making minimal demands. The exception may be in cases where law enforcement becomes involved; if there is a delay in the campaign schedule due to waiting for law enforcement to act, it could be the case that staff run out of work that can be done before law enforcement finish their action.

How many hours per week does the typical staff member work on the campaign when it is making its maximum demands?

1

2

3

4

5

6

7

1= 1 or fewer
7=80 or more

Notes: Because campaigns vary so much, there’s no estimate for even a single role (e.g. investigator) that would be appropriate for all campaign types. In most cases, not all participants will be actively working on the campaign at all times, but some staff member will be working intensely on the investigation at certain times, such as when the investigator is on the farm filming or when it is necessary to review many hours of footage from the investigation. This could easily take 40 or more hours per week (and certainly does, in the case of investigators who secure jobs on farms and must review video they have taken daily in addition to that work).

Note that if the campaign involves several different types of staff roles with different time demands, it may be helpful to answer the above questions for each role rather than for the typical staff member.

Evaluation for III.2.c

How many staff members are involved with the campaign when it is ongoing but at the time when fewest people are working on it?

1

2

3

4

5

6

7

1=It runs automatically with at most one person supervising it
7=Over 50

Notes: For most campaigns, the time when the fewest people are involved is probably either during the research and planning stages or after the release of the video, when one or more campaign spokespeople are available to speak with the press and the campaign is being promoted on social media (possibly by the same people) but all major campaign materials have been developed and edited.

How many staff members are involved with the campaign at the time when the most people are working on it?

1

2

3

4

5

6

7

1=It runs automatically with at most one person supervising it
7=Over 50

Notes: Most campaigns have some stage when several people are actively working on the campaign, possibly in different areas. For instance, in an open rescue or investigation, the investigation team may consist of several people. Or in an undercover investigation, the investigator may be taking video and notes, while other staff also review the video to determine when enough material has been accumulated, research the farm involved, and begin setting up a campaign website and materials.

Evaluation for III.3

How many rare or difficult-to-obtain materials does the campaign require?

1

2

3

4

5

6

7

1= None
7=Several which are crucial to the success of the campaign

Notes: Some campaigns do not involve any materials which are particularly difficult to obtain. Undercover investigations in which cameras used must be hidden require specialized camera equipment that is generally produced for law enforcement. It’s possible for advocates to purchase this equipment, but it is expensive and organizations would not otherwise need it.

How severe is the difficulty of obtaining the hardest-to-obtain material needed for the campaign?

1

2

3

4

5

6

7

1= There are no materials which will be difficult to obtain
7=There may be an insuperable obstacle to obtaining this material

Notes: Specialized camera equipment would be difficult to obtain for a small organization with a very limited budget, but for medium or large organizations the cost is reasonable. (In the thousands of dollars.)

What resources does the campaign require that may be difficult to obtain?

Undercover investigations in which investigators must film secretly (e.g. investigations in the US where investigators are employed by the farms they’re investigating) require specialized camera equipment similar (often identical) to that used by undercover law enforcement officers.

Summary

General Summary of Area III (Barriers to Entry). Summary may include relevant comments about the following topics: the level of complexity of the skill sets needed for the campaign; the intensity of the work; and the feasibility of acquiring needed material resources

Undercover investigations that involve taking a job at an animal agriculture facility involve a substantial amount of resources. Resources can include hiring and training an investigator to do this work, having the proper recording equipment, seeking legal advice regarding the investigation, having the investigator undergo the emotional and physical strain of their work, having staff comb through the footage, and having the ability to share the footage with media outlets in a professional and effective manner.

Open investigations and rescues, or other similar activities that don’t involve employment at the facility, can probably be done with more limited resources. However, the basic requirements of a capable investigator (or several), recording equipment, and careful planning still apply. These costs can be reduced if the action is taken as part of an existing organization or network.

Several of these requirements seem to reduce as the organization conducting the investigations becomes more experienced. For example, recording equipment can be used for multiple investigations. Importantly, once an organization has established itself, getting attention from the press likely becomes much easier.

The intensity of work on a particular investigation can vary widely. The investigator might have to visit several facilities to find one where they can film. They might need to work in a position that doesn’t involve direct contact with the animals for a lengthy period of time before moving into a position where they can document cruelty.

Error Tracking

What differentiates the campaign from past similar campaigns? Could any of these factors affect the difficulty of achieving the campaign’s goals? Keep in mind that it may be difficult to ascertain which factors may have contributed to a past campaign’s success/failure.

Campaigns are differentiated primarily by their goals and tactics and the organizations carrying them out. These do clearly affect success rates, though not necessarily resources used, so for a given campaign it is important to pay attention to which past examples are used for comparison.

How rigorously documented were previous campaigns? Could estimates of staff time spent etc. be in error?

Average estimates of staff time spent are likely highly accurate, as they are based on the experience of organizations who have conducted multiple campaigns. Because staff can work on several campaigns at once and may not track how much time is spent on each, estimates of the variability have more room for error.

Have environmental factors changed which would affect either the skills required to conduct the campaign or the difficulty of obtaining necessary materials?

There are many examples of recent campaigns, so it is not likely that significant environmental changes have occurred between these campaigns and campaigns currently being conducted. Some U.S. states have adopted “ag gag” laws that make undercover investigations more difficult. Currently, organizations have been able to continue conducting investigations, but this could change depending on whether more laws come into effect and whether legal challenges to these laws, largely on the basis of freedom of speech, are successful.

Area IV: Expected Indirect Effects

Oftentimes, a campaign will have effects that reach beyond its intended goals. These may be positive (i.e. in-sync with the ethics driving the campaign) or negative (i.e. working against the change effected by the campaign). There are multiple factors to consider when evaluating external influence. These effects are especially difficult to estimate precisely because they are often diffuse and because even when campaigns have carefully tracked their progress towards their explicit goals, they may not have attempted to track other effects. However, diffuse and indirect effects of some campaigns may be significant parts of the campaign outcome, so we consider any evidence about these effects as a factor in evaluating interventions.

IV.1 Positive influence

Good externalities may come from a campaign.

IV.1.a Chain effect

A chain effect is when those affected by a campaign spread the message of the campaign and thus extend the influence of the campaign’s message. For example, an online advertising campaign might have a particularly high positive chain effect, due to the ease with which a viewer can share a website or video link with others.

IV.1.b Benevolent slippery slope

It is possible that the success of the campaign will indirectly inspire other positive victories due to social pressure, or desire to imitate. For example, if multiple states enacted bans on gestation crates, some pork producers in other states might bow to public pressure and voluntarily phase them out themselves.

IV.1.c General education

General education is when a campaign instills a certain mindset in some subsection of the public that will go on to produce beneficial outcomes. For example, a campaign to oppose “ag-gag” laws laws might cause some people to develop a negative attitude towards animal agriculture in general, regardless of whether or not the campaign itself is successful.

IV.2 Negative influence

Bad externalities may also arise from an intervention

IV.2.a Alienation

An imperfect campaign may run the risk of turning some potential supporters away from the cause and/or inciting a backlash against the goals of the campaign. For example, a public awareness campaign that uses offensive imagery to make its point may risk alienating potential supporters and/or fueling the influence and credibility of its detractors.

IV.2.b More harm than good

There is also be a chance that the intended results of a campaign might actually be more harmful than beneficial to animals. This is a difficult-to-spot type of negative externality because it requires researchers to recognize when something is being overlooked in how a certain abusive system would respond to a successful attack against it. For example, a campaign against the consumption of beef might merely push people to consume more chicken (which would be particularly bad as it takes the meat of roughly 200 chickens to equal the same amount of meat from a single cow).

IV.3 Evaluation and Error tracking for “External Influence”

Evaluation for IV.1.a

How large a chain effect does the intervention likely have?

1

2

3

4

5

6

7

1= No reason to expect a chain effect
7=Campaigns often go viral, with a high percentage of people who see hear the message sharing it in close to the original form

Notes: Videos from investigations and rescues are some of the most viral content that farm animal advocacy organizations produce, especially in cases where the video includes new or surprising content. When users share the videos, others see exactly the message the organization intended to disseminate.

Evaluation for IV.1.b

How likely is the campaign to inspire other victories?

1

2

3

4

5

6

7

1= No reason to expect the campaign to inspire indirect victories
7= Most similar campaigns have inspired successful imitations or victories beyond their expected scope

Notes: Video investigations and rescues often provide information that inspires other campaigns or is instrumental to them. Some organizations’ investigations and rescues have also prompted similar work by other groups.

Comment on any evidence (especially taken from observations of past campaigns) suggesting ways in which this intervention may directly or indirectly inspire other activism movements.

For instance, Direct Action Everywhere’s open rescue campaign is directly inspired by Animal Equality’s investigations and rescues and by the work of Patty Marks. In the US it is common for undercover investigations to be used as part of corporate advocacy campaigns (sometimes by other organizations than conducted the investigation). And investigation footage is used in many short films and documentaries like Farm to Fridge and Death on a Factory Farm, which are widely distributed even long after the investigations involved are no longer news.

Evaluation for IV.1.c

How significant a change in public mindset is this campaign likely to cause?

1

2

3

4

5

6

7

1= No change; the public will be entirely unaware of or uninterested in the campaign
7=A significant change; the campaign will reach most of the population with a message that is highly contagious and represents a significant departure from current norms

Notes: The campaigns will likely reach a large number of people, but many will receive a message that is not much different from that already promoted by society (it is unacceptable to be cruel to animals, but not necessarily to farm them). While some may receive the message that all farming is unethical (or similar messages very unlike current norms), there isn’t evidence that these messages are passed very contagiously from video investigations. Some people will come to see animal agriculture as a bad thing because they come to understand there is a large amount of suffering involved of which they were previously unaware or which they had been able to ignore.

Evaluation for IV.2.a

How much alienation is this campaign likely to cause?

1

2

3

4

5

6

7

1= None; this campaign has no apparent controversial features
7=Significant alienation; a large contingent of potential allies will likely take offense at some aspect of the campaign, and the campaign may be perceived as representing the overall position of a large number of animal advocates

Notes: Most video investigation/rescue campaigns are popular with the general public and unpopular only with industrial producers of animal products, who are not natural allies of animal advocates.

Comment on the ways in which this campaign might alienate certain members of the public. Specifically, who might be alienated, and how damaging might the alienation be, in terms of negative impressions carrying over to other similar organizations?

Most investigations do not have a high risk of alienating members of the general public, as the general public tends to agree that the treatment they depict is problematic. There is some risk of alienating companies which are working towards implementing better policies by making them the target of an investigation (especially since many food companies do not distinguish between animal advocacy group, so relations with one organization might be damaged by another organization doing an investigation). Advocacy groups can and often do reduce this risk by focusing an investigation on another company if they know that one is currently revising or recently has revised a policy; even once a farm has been selected, there are usually several possible policy targets, including the farm itself, productions companies, and distributors like restaurants and supermarkets.

Some specific types of investigation or rescue may be more controversial and run a greater risk of alienating members of the public. For instance, campaigns directed against companies or farms known for their “humane” production policies may encounter public resistance because they seem to be going after wrong or innocent targets, especially to people who (like most consumers) are not aware of the actual regulations behind labels such as “humanely raised” or “natural”. They may also, like other campaigns, result in alienation from the companies targeted. However, the more invested in “humane agriculture” the target of the campaign is, the more likely they are able to distinguish between animal advocacy groups and will not assume that the actions of the campaign represent the opinions of all animal advocates. Companies very invested in having a “humane” image are likely to have already had significant positive contact with animal advocates while developing that image.

Evaluation for IV.2.b

How high is the risk that the campaign may be more harmful than beneficial?

1

2

3

4

5

6

7

1= Extremely low risk; the campaign will have few, highly controlled effects
7=High risk; the campaign ignores significant parts of the system it is working in that pose a threat to animals

Notes: Video investigations and rescues do not generally seek to be the direct cause of specific changes; rather, they seek to increase awareness and concern about how animals are treated, especially in industrial agriculture. They are closely connected to corporate outreach work, which does seek to make direct changes; see our evaluation of corporate outreach for a discussion of risks from that activity.

The more indirect effects of investigations and rescues are harder to predict, simply because they are more indirect. But media coverage of the meat industry (of any kind) has been linked to reduced meat consumption, so it seems very likely that media coverage in response to investigations and rescues has positive short term effects on meat consumption. The longer-term effects of exposure to the messages of the videos seem likely to depend on the particular message chosen and to be similar to hearing the same message through other channels. Concerns some advocates have about particular messages include that welfare messages may normalize animal use, that animal rights messages or presentation may be radical or alienating, and that any message which successfully causes the recipient to care about farmed animals may also open them to caring more about the environment in itself (vs. as it affects animal or human happiness), which could be bad from the perspective of wild animal suffering. In our opinion, it is likely that all messages distributed through investigation videos do more good than harm, in that their primary effect seems likely to be either educational or in reducing speciesism toward farmed animals, though videos likely vary in efficacy with regard to both these aims.

What specific practices does the intervention seek to change? Would the replacement practices necessarily be an improvement for the animals involved?

Campaigns sometimes seek to change specific practices on farms, including workers demonstrating illegal cruelty to animals and systems of confinement. As we concluded in our evaluation of corporate outreach, the specific policy targets of these campaigns are usually clearly beneficial to the animals involved; when they aren’t, the replacement practices seem no worse and at least probably better.

Campaigns also seek to reduce consumption of animal products, which leads to fewer animals being bred and raised for food production. Most animal advocates see this as beneficial, because farmed animals live in circumstances which appear to be extremely unpleasant and demonstrate a variety of stress behaviors in response to these conditions. However, it should be noted that population ethics is difficult even when applied to humans who can actually express a preference between their current circumstances and never having been born, and there is necessarily more uncertainty with regard to the preferences of animals who cannot communicate about abstract topics.

Summary

General Summary of Area IV (Expected Indirect Effects).Summary may include relevant comments about the following topics: indirect reactions to the campaign (positive and negative); and long-term externalities of the campaign (positive and negative)

Undercover investigations probably have some indirect effects through inspiring other organizations to conduct similar campaigns and providing an environment in which it is easier for future campaigns to gain more attention, although there are concerns about saturation at a certain number of investigations in a specific country or region.

We think the indirect effects are fairly large compared to most other interventions, mostly because of the press coverage they have received and how frequently they are referenced by the general public when farmed animals issues are brought up. The coverage of “ag gag” laws, for example, seems to have helped the general public appreciate the importance of helping farmed animals.

Error Tracking

What assumptions have been made?

Our reasoning in this section is based largely on intuition and speculation due to limited empirical evidence, but we have tried to account for all possible effects rather than making definitive assumptions.

How reliable is the data regarding past campaigns?

We do not have data on the long-term effects of any specific campaign, but we can directly observe many outcomes of interest and estimate how those effects relate to these campaigns as a whole, such as over the past decade.

How reliable is data predicting expected responses to various campaign tactics?

It seems possible to predict levels of media coverage and, to a lesser extent, response from the corporations involved. It seems much harder to predict long-term or indirect outcomes.

Area V: Expected Direct Effects and Overall Efficiency Analysis

This area of evaluation is designed to produce a single ratio of dollars spent per unit of suffering reduced or per number of animal deaths averted. We believe this is a useful factor to consider when planning an intervention, as different types of campaigns can have vastly different cost/benefit ratios. Given the limited amount of funds available to an organization of any size, this section explores ways of making sure that those resources are used optimally. We break down this calculation into component expenditures and results. If we could rely entirely on the results of this section, for many purposes we could omit the other sections of the evaluation entirely. However, in reality our estimates of both costs and expected results will be fallible, so reasoning about an intervention’s likely effectiveness from other perspectives (as above) allows us to be more confident of our conclusions than we could be about a direct efficiency analysis alone.

V.1 Expenditures

A successful intervention requires the input of resources, both financial and otherwise. It is important to understand and quantify the full extent of the costs to run a campaign.

V.1.a Monetary costs

Monetary costs include both upfront costs to begin an intervention and maintenance costs to sustain an intervention over time. Upfront costs are expenditures that go towards things such as the recruitment of volunteers, the purchase of office supplies, consulting fees, etc. Such expenses can be a limiting factor for an intervention if insufficient liquid capital is available. Maintenance costs are expenditures that are expressed in the form of dollars per unit time or per event. They include things such as material expenses (e.g. pamphlets, gas for vehicles, etc.) and money allocated to train and compensate workers (e.g. salaries for staff, costs involved in educating staff, costs involved in training volunteers for an event, etc.). Maintenance costs also involve non-specific overhead, such as office leases, travel expenses, and salaries/training costs that are not specific to any one event within a campaign. The amount of monetary resources needed for a campaign directly relates to how long a campaign can be sustained for and whether it can even be undertaken in the first place.

V.1.b Personnel

Personnel includes the number of people needed and the level of expertise needed to run an intervention. The number of people needed includes staff, long-term volunteers, single-event volunteers, outside help, consultants, and media coverage by animal activist journalists. While the cost of salaries is covered under “monetary” expenses, there is a clear opportunity cost of using up the time of people who are committed to working on animal activism projects. A poorly planned campaign could pull activists away from other, more effective interventions. The level of expertise required from the personnel includes both pre-existing expertise and the amount of training it is necessary to provide to new recruits. This could range from little to no expertise (e.g. one-off volunteers recruited for discrete events), to high expertise in animal law, high expertise in policy, high expertise in campaign management, high expertise in data analysis, etc. Recruiting the services of a person with a high level of expertise relevant to animal activism carries a higher opportunity cost, given that person’s theoretical ability to do more for a different activism campaign than the average activist.

V.1.c Time

Time refers to the time expenditure by an intervention’s staff, including both time committed during the intervention and time committed to pre-campaign training and planning. The former category involves such things as hourly shifts at events, weekly staff time, one-off consulting appointments, etc. This again is proportional to the magnitude of opportunity cost. TIme committed before an intervention begins includes time for event preparation, market research, experiment conducting, data collection, etc. This is time invested in laying the groundwork for a campaign. A campaign with a very short startup time may in some cases be preferable to a campaign that has slightly more potential effectiveness per unit time, but which also has a long startup time.

V.1.d Unknowns

Unknowns describes the category of unplanned expenses. While unknown expenses are by definition unexpected, it is possible to attempt to consider a range of possible emergency expenses and multiply the cost of each by the likelihood of the unplanned event occurring. This category includes such expenses as legal payments from being sued by a group or person, replacement costs for event materials lost on-site, etc.

V.2 Results (“Profits”)

After estimating expenditure of resources, we estimate the expected “return on investment,” measured by the degree of success a campaign achieves in reducing animal suffering or saving animal lives.

V.2.a Breadth

The breadth of an intervention’s results is defined as the number of people reached by the campaign. It can be expressed as the product of the number of people contacted per event or per time, and the number of events or units of time within the intervention. It is important to understand what is meant by the phrase “reached by the campaign.” A person does not have to be directly contacted by a staff member in order to be “reached”. A legal campaign to change the way a certain farm animal is raised would (if successful) “reach” every person who consumes that animal product.

V.2.b Depth

Depth measures the extent of impact upon animal welfare the average person reached by an intervention will have. This is expressed as the product of multiple terms: the percent of people contacted that end up reducing animal product use (this is to be broadly construed as any lifestyle change that reduces a person’s negative impact on animal welfare, either by completely abstaining from the use of certain animal products or by switching to more humane animal use infrastructures); the amount of time a person who reduces animal product use continues that reduction; the number of animals affected by the animal product use reduction; and the improvement in welfare for each of those animals affected. Each of these factors will here be defined.

The percent of people reducing animal product use is the fraction of people reached by an intervention that make certain measurable changes to their lifestyles (whether voluntarily, as in a dietary change, or involuntarily, as in consuming animals that are raised more humanely under new laws). There may be multiple categories of animal use reduction, with different accompanying conversion rates. For example, from the group of all people reached by an intervention, it might be found that 1% converted to veganism, 6% lowered their overall chicken consumption by a fifth, and 10% will only purchase cage-free eggs.

The amount of time a given lifestyle is adopted is defined as the length of time before a person adopting a new lifestyle is expected to experience recidivism back to their old lifestyle or back to a less humane lifestyle. This time may be different for each type of lifestyle change. The net good done by any given convert is proportional to the amount of time they stick with their newly adopted habits.

The number of animals affected by animal product use reduction measures the number of animals that are expected to have their welfare improved by a lifestyle change made by a person reached by an intervention. Again, this number might be different for each type of lifestyle change. This compensates for the fact that reducing, for example, an average omnivore’s beef intake has a much weaker effect than reducing the same person’s chicken intake, given the relative values of pounds of food produced per animal for cows versus chickens.

Finally, improvement in welfare per animal affected quantifies the decrease in suffering per animal affected by the campaign. Not all changes to an animal’s life are equal. For example, there needs to be a way to compare and rank the gain associated with a chicken being moved from a battery cage to an open pasture with the gain associated with a less painful slaughter method being used on a pig. This is a complex determination. It takes into consideration the improvement in living conditions for each animal, the mental complexity of each animal (i.e. their ability to experience those living changes as ethically meaningful increases in mental well-being), and the number of animals saved from slaughter.

*V.2.b.2 Notes about improvements in welfare

The determination of the variable “improvement in welfare” also requires a decision regarding how to weigh the unit of “lives saved” against “lives improved”. Some animal activists take “reduction of suffering” as their bottom line, and value saving a life only so far as this prevents the suffering associated with animal death. Other activists also include the increase in pleasure as a factor in calculating animal welfare; they would regard lives saved as valuable both for the avoidance of the pain of death and for the gain of life’s pleasures available to an animal spared from death. Finally, some animal activists find intrinsic value in preventing animal deaths, regardless of suffering averted or pleasure increased.

The determination of suffering reduced requires some sort of an understanding of animal neurology, as discussed above in the form of “mental complexity.” This is a difficult variable to quantify, but remains incredibly important to the overall assessment, since less sensitive (with respect to the ability to experience pain) animals should be given less ethical weight compared to more sensitive animals. It is our hope that our ability to rank the welfare of various animals will improve as research into nonhuman animal consciousness progresses. For now, this crucial variable remains a grey area within the analysis.

V.3 Evaluation

Please comment on what has been empirically observed regarding the effect of relevant human lifestyle changes on animal welfare and on what has yet to be empirically determined.

Results from two studies trying to study the effect of media coverage of animal welfare on short term demand for animal products were inconclusive, but if one believes they do reduce short term demand, video investigations are among activists’ most reliable ways of securing media coverage of animal welfare.

Other effects, such as corporate policy change and individual long-term dietary changes, are less well-established as resulting from video investigations and rescues, in the first place because they often arise without a direct connection to an investigation (and so the counterfactual impact of investigations is not clear) and in the second case because few efforts have been made to directly connect exposure to an investigation to dietary change.

Evaluation for V.1.a

What upfront and maintenance costs does the intervention have, including costs of supplies, salaries, and office space? How consistent are these costs between previous campaigns?

Upfront costs can include stock in the target corporation or a campaign website and video. Rarely, they may include both types of resources. Campaign set-up time is accounted for in the estimates of weekly workload we received from our sources, so we included minimal staff and recruitment time as upfront costs, folding in staff time spent researching the target company etc. into the maintenance staff costs.

Maintenance costs usually include salary and office space for one worker, shared among 2-50 ongoing campaigns at a given time, and (depending on the organization) also among other campaigns not fitting the corporate outreach model. They may also include web hosting or occasional travel to meet in person with corporate executives or stockholders.

Upfront costs vary widely between campaigns, depending on whether there is a need to purchase stock and on whether a website and video are produced and if so, by a volunteer or paid contractor. Maintenance costs are fairly similar between campaigns, as far as we can understand.

V.3.1 Calculating Efficiency

We offer a way to calculate as objectively as possible the efficiency for a given campaign measured in the units of animal suffering or animal deaths avoided per US dollar equivalent expended.

Unit of intervention length

investigation/rescue

a1

How the length of the intervention is measured. E.g. “days”, “weeks”, “number of events”

Estimated length of intervention, in intervention units (a1)

1

a2

Use unit defined in a1

Expenditures (Unit = US$)

Pessimistic (Highest) Estimate

Realistic Estimate

Optimistic (Lowest) Estimate

Based on ACE’s 2014 evaluation of MFA

Based on an open rescue paying some staff

Based on an open investigation paying some staff

Cost of material resources

10,000

2,250

100

b1

Supplies, leases, consulting fees, etc.

Cost of recruitment and training

2,000

50

0

b2

Advertising, value of time spent interviewing, etc.

Personnel costs

72,911

8,500

4,750

b3

Salaries, etc.

Expenditures total

84,911

10,800

4,850

d1

=b4+c3*a2

Pessimistic (Lowest) Estimate

Realistic Estimate

Optimistic (Highest) Estimate

Based on economic data regarding media coverage of treatment of pigs

Based on economic data regarding media coverage of treatment of chickens

Based on treating each view as a view of an online ad

Unit of suffering

Years of farmed captivity averted, or equivalent suffering averted

e1

The unit by which the results of an intervention are measured. This may be “animal lives saved”, “years or year equivalents of a factory farmed hen’s life averted” (see V.2 and V.4 in evaluation guidelines), “years of farmed captivity averted”, or something different.

Direct suffering avoided per intervention unit

12,858

2,249,658

0

f1

Measured in terms of unit of e1, for all direct results of an intervention (e.g. directly negotiating for the release of an animal from a factory farm)

Indirect suffering avoided

Number of people reached by campaign per intervention unit

9,123,077

g1

A person does not have to be directly contacted by a staff member in order to be “reached”. They must merely encounter the campaign in some capacity, including living under a legal jurisdiction being targeted by a legislative campaign.

Proportion of people contacted likely to adopt vegetarianism

.021

g2.1

The expected percentage of people reached by the campaign that adopt vegetarianism

Proportion of people contacted likely to adopt veganism

.007

g2.2

Same as for g2.1, but for veganism

Indirect suffering avoided per person contacted

2.07

g3

Multiply g2.1, g2.2, …, g2.n each by their respective “lifestyle multipliers” (see chart below) and then sum the resulting products

Results Total

12,858

2,249,658

3,980,769

h1

=f1+g1*g3*a2

Final Total: the proposed intervention has a calculated efficiency of h1/d1, for a campaign of 20 weeks, with results being measured in years of farmed captivity (or equivalent suffering) averted.

Pessimistic: .15 lives/dollar

Realistic: 210 lives/dollar

Optimistic: 820 lives/dollar

Lifestyle

Mean Years of Retention (i.e. years before a person is expected to no longer follow an adopted lifestyle>

Lives Saved per Year (based on ACE evaluations)

LIFESTYLE MULTIPLIER (total units of suffering avoided)

Vegetarianism

7.03

10.35

72.74

Veganism

7.03

11.08

77.87

V.4 Calculation Sheet Discussion

A few points on the sheet:

To express the magnitude of animal suffering avoided within a formula, it is necessary to convert to a common unit. It is difficult to express this as an objectively defined value, so the suggested way to measure the results of a campaign is to list the types and number of animals affected, and the specific ways in which their suffering is expected to be reduced.Ideally, the various forms of suffering prevented by a campaign could all be converted to a common unit. The unit for “results” (i.e. measurement of campaign success) has been defined here as years of factory-farm level suffering (i.e. the amount of suffering experienced across one year on a factory farm) averted. This is a crude unit, as it does not allow for distinctions to be made between different types of animals (with different types of consciousness), and makes it hard to deal with suffering that happens outside of the factory farm industry. It should also be noted that the calculations in the “lifestyle multiplier” chart within the calculation sheet only takes into consideration farmed fish, not wild-caught fish.If an alternative unit is desired, one “YBHS” (“years of battery hen suffering”) could be defined as the negative utility produced by keeping an average hen in a battery cage for one year. A year of non-battery conditions for hens could be estimated as a factor of the YBHS (e.g. it has been suggested in the Wageningen Journal of Life Sciences (Del Mol, et al., 2006) that cage-free facilities reduce the suffering of an egg-laying hen by half; thus, a year of cage-free hen suffering prevented would be equivalent to only 0.5 YBHS avoided). Other non-hen animals could have their yearly suffering also calculated as a factor of the YBHS, after taking into consideration both the nature of their environment and the capacity for the animals to feel pain relative to that of a hen.This is perhaps a more controversial method of calculation, as it requires making estimations about the relative abilities of different animals to experience suffering. While it is clear that the expected suffering of, say, a gestation crate-confined pig is many orders of magnitude stronger than that of a fly caged in a jar, it is not as clear that the same pig should be given greater weight than a battery-caged hen. However, estimations must be made for the purposes of this calculation. To count all animals as equal is itself an estimation, so the decision cannot be easily avoided.

The expenditures/results multiplier sections include somewhat arbitrary (though intuitively reasonable) coefficient values. This is an attempt to quantify non-directly-calculable variables, in order to adjust the final values accordingly. As more research is done and more information is known about specific campaigns, these coefficients could change dramatically.

The “total expenditures” and “total results” boxes (f1 and m1) give values specific to a campaign of the length described in a2. It is theoretically scalable to any campaign length, though unless upfront costs scale with campaign length, accuracy will be improved by revising the calculation to address the new length directly.

The final output is given in terms of unit of animal welfare increase per cost in US dollars.

The “lifestyle multiplier” key refers to the number of years of factory-farm level animal suffering are expected to be averted due to a person beginning to adopt a certain lifestyle (See I.2.b for more details). This is calculated by: [mean years person is expected to maintain lifestyle] * [number of years of animal suffering avoided per year of lifestyle].

V.5 Error tracking for “Efficiency”

How reliable is the scientific data on the causal relationship between campaigns and behavioral changes?

The scientific studies on media coverage’s relationship to animal product demand are somewhat reliable, but the results were ultimately inconclusive.

How reliable is the scientific data on the causal relationship between behavioral changes and improvements in animal welfare?

We estimate the impact of dietary change on animal welfare in the Leafleting and Ads Calculators on our website. Our certainty regarding whether changes in diet lead to improvements for animals is much greater than our certainty for whether the dietary changes occur, and to what degree.

How much direct knowledge exists about the costs and outcomes from past campaigns?

We have substantial knowledge of the costs of these campaigns, and a significant amount of knowledge of the immediate outcomes of media coverage and corporate policy change.

How reliable are budget reports from past, similar campaigns? Could budgets mistakenly fail to list some costs and/or record unrelated expenses as campaign costs? Could some budgets fail to temporally synchronize expenditures with results?

The budget information seems very reliable relative to the outcome information, although there are some difficulties in, for example, assessing future campaign costs based on previous campaign costs.

Has there been significant economic or social change since the time of past, similar campaigns that might invalidate their quantification of costs/outcomes? What other changes have happened (such as with supply chains for campaign resources) since past, similar campaigns?

There haven’t been major concrete changes, although the introduction of ag-gag bills does mean investigatory organizations have a little more difficulty in their work. There is also a possibility that marginal press coverage could have different effects because the public has already been exposed to similar information.

How reliable are estimates about things like the number of people expected to attend a campaign event, the number of campaign events to be held within a given time span, etc.?