Working groups considering fundamental questions concerning the pursuit of transparency in qualitative empirical research, which cut across the particular forms of research in which qualitative scholars engage

The focus of this working group is on evidence that comes from first-hand observations of, or interactions with human participants, including formal as well as informal, unstructured interviews; observation of/participation in meetings/events; and non-interview interactions with human subjects, including surveys. (Ethnography is the focus of a separate working group.) This working group will, in particular, focus on two potential types of transparency with such evidence: transparency about how scholars have made observations or generated evidence through research with human participants; and questions of when, why, and how this evidence can or should be made available or easily findable to others.

This working group's deliberations will consider the circumstances under which, and the reasons why, researchers might share elements of their interactions with human participants; why, when, and how researchers can/should be transparent about the process of collecting interview and survey evidence or about the nature of their engagements with research participants; the costs of and limits to transparency in these areas; and ways of being transparent with research participants themselves. This working group will also investigate and assess technologies and infrastructure that might aid scholars wishing to share evidence from first-hand observations of, or interactions with, human participants.

To download the working group's draft report, select the "DRAFT REPORT" announcement. Please provide comments or other feedback on the draft via the first topic-thread "Comments on Draft Report ..." You may also continue to view and add to the earlier threads. Please log in first to have your post be attributable to you. Anonymous posts will display only after a delay to allow for administrator review. Contributors agree to the QTD Terms of Use.

From the Introduction: This working group will, in particular, focus on two potential types of transparency with ... evidence: [1] transparency about how scholars have made observations or generated evidence through research with human participants; and [2] questions of when, why, and how this evidence can or should be made available or easily findable to others.

The first point is methodological. The second point is ethical. I would like to comment on the first point.

I think it is important to emphasize that the word “transparency” has different implications for the methods of quantitativists and qualitativists/interpretivists.

As a quantitativist ideal, transparency assumes that a collection of facts or statistics can be stored so that another researcher can use them to test the inferences drawn therefrom, or to replicate the method by which the data was aggregated. Replication is presumed to be the key to scientific validation of reported research results. But the association, almost equation, of transparency with replicability is generally inappropriate for qualitative/interpretive studies.

If having a methodological appendix is to be considered “transparent,” and therefore “good science,” then that requirement could reinforce the presumed importance of the quantitative model of science to which it belongs. But that model of science and its implicit configuration of methodological values is a threat to the scientific status of qualitative/interpretivist approaches.

While “transparency” as a matter of professional ethics is important to discuss and to clarify, the discussion of methodological transparency should include the warning that no endorsement of replication is intended.

The deeper problem here isn’t that journals requiring a methodological appendix should also have a condescending exception for submissions made with appeals to privacy concerns. That exception would only preserve the implication that qualitative/interpretivist approaches are less scientific than the quantitative approaches – like a tolerated step child of the real political science.

An appendix discussing “how scholars have made observations or generated evidence through research with human participants” should not be understood to honor the positivistic ideal of replication as the key to validity. Such studies cannot be replicated. Each is a unique product of the political scientist who produced it, in one-of-a-kind circumstances.*

Replication as a general value for political science is misplaced and self-defeating for qualitative/interpretivist science. While it may be appropriate for some kinds of quantitative studies, it is inappropriate for unique interpretive studies. Of course, methods for validating claims to knowledge are crucial for any scientific study. The methods for qualitative/interpretivist studies can be criticized on procedural or methodological grounds, and the conclusions offered by such studies can be challenged by logical analysis and/or the results of other participant observer studies. But validity depends on expert consensus, not on replicability.