Nonprofit organizations and program evaluators are increasingly being called upon to evaluate multi-stakeholder initiatives. These initiatives often depend upon collaboration among various organizations and agencies. As a consequence, the need to evaluate collaborations/coalitions has become an important requirement for a wide-range of contemporary program evaluations.

In a recent article, “Evaluating Collaboration for Effectiveness: Conceptualization and Measurement,” (American Journal of Evaluation, 2015, Vol. 36, p. 67-85) Lydia I. Marek, Donna-Jean Brock and Jyoti Savla discuss the customary difficulties in assessing collaborations, including, “lack of validated tools, undefined or immeasurable community outcomes, the dynamic nature of coalitions, and the length of time typical for interventions to effect community outcomes…the diversity and complexity of collaborations and the increasingly intricate political and organizational structures that pose challenges for evaluation design” (p.68).

Building on previous research by Mattesich and Monesy (Collaboration: What Makes it Work? Fieldstone Alliance, 1992) which outlines a six category inventory of characteristics typical of collaborations ( i.e., environment, membership characteristics, process/structure, communication, purpose, and resources), Marek, et al., argue for seven key factors that typify successful inter-organizational collaborations and coalitions:

1. Context—the shared history among coalition partners, the context in which they function, and the coalition’s role within the community

Marek et al. offer a tool for assessing the effectiveness of collaboration, a 69 item Collaboration Assessment Tool (CAT) survey instrument, which asks a series of questions for each of the factors identified above. For example, the “function” domain of the survey asks organizational respondents the following questions:

This coalition has clearly defined the problem that it wished to address

The goals and objectives of the coalition are based upon key community needs

This coalition has clearly defined short-term goals and objectives

This coalition has clearly defined long-term goals and objectives

Members agree upon the goals and objectives

The goals and objectives set for this coalition can be realistically attained

Members view themselves as interdependent in achieving the goals and objectives of this coalition

The goals and objectives of this coalition differ, at least in part, from the goals and objectives of each of the coalition members

“Evaluating Collaboration for Effectiveness: Conceptualizations and Measurement,” offers a valuable tool for assessing the often complex inter-organizational relationships that constitute multi-stakeholder collaborations. The CAT offers a system for gathering data for evaluating the quality of collaborations, and implicitly suggests an outline of the key characteristics of successful collaborations. The CAT tool will be useful for organizations who are embarking on collaborations, and for program evaluators who are charged with evaluating the success of such inter-organizational collaborations. To learn about our evaluation methods visit our Data collection & Outcome measurement page.