DA-RT Symposium

The Spring 2016 issue of the Comparative Politics Newsletter contains a symposium on Data Access and Research Transparency (DA-RT). This page exists to allow scholars to respond to the contributions in the symposium. If you wish to submit a response, please contact the Newsletter editors, Sona Golder and Matt Golder. Send your comments to sgolder@psu.edu or mgolder@psu.edu. Responses to the symposium contributions will be published unedited within 24 hours of submission. Responses will appear below this paragraph and in chronological order. To ensure a high-quality and polite dialogue, we will require that responders provide their name and affiliation. The names and affiliations of responders will be published along with their responses.

(Posted in reverse order received.)

Response from Robert Kaufman
(Rutgers University)

I believe that the debate over DA-RT has been a very healthy one, which has been joined with professionalism and good will by participants on all sides of the issue. This spirit was very well reflected in the recent symposium in the Comparative Politics Newsletter, and I would like to weigh in with a few thoughts of my own. At the outset of the debate about DA-RT, I was inclined to be sympathetic to the effort to specify the norms of knowledge production in our discipline, and I believed (and still do) in the sincerity of promises to respect the diverse research traditions in comparative politics. Upon further reflection, however, I have come around to the view, expressed by several critics who participated in the CP symposium and in the contributions by the editors of CPS and World Politics, that the costs of the enterprise outweigh the potential benefits.

In principle, the DA-RT initiative pursues the laudable goal of making the empirical bases of knowledge claims clear and available to other researchers. Moreover, as several participants have noted, providing a public accounting of the data and methods should be an appropriate step for quantitative researchers, although there is some room for legitimate debate about the time frame in which original data sets should be made available to others. The problem comes with proposals to extend and adapt these norms to the various approaches to qualitative research. Efforts to do so may be both possible and desirable in some instances, but the professional disincentives, ethical limits, and practical costs – particularly to young researchers – can be overwhelming.

To me, the most critical stumbling block is the Journal Editors Statement (JETS). It is one thing for members of the Association to debate and even try to frame norms of data access and research transparency. It is quite another for journal editors to add another filter to the already demanding gauntlet of peer reviews and editorial judgments that a paper must survive in order to reach publication. Editors, to be sure, do serve as gatekeepers, as Deborah Yashar has noted, and they have an obligation to insist on rigorous reviews. But it is far from clear that adding new requirements to the review process – even ones that are flexible and respectful of alternative research traditions – will improve on existing practices. The great risk, of course, is that it will result in a more limited and less creative research environment.

This risk is especially high when there is so little agreement about the specific standards that might be generally applicable across the research communities within comparative politics. The evidentiary standards for arriving at and documenting inter-subjective claims need to emerge through inter-subjective processes. Such processes, to be sure, are shaped by the hierarchical structure of our profession – and probably should be. Still, for the time being, it seems prudent to rely on the norms of research transparency and data accessibility that emerge from within the diverse research communities themselves, rather than ask journal editors and their boards to elaborate additional hoops for authors to pass through.

by Robert Kaufman
Rutgers University

Response from Nancy J. Hirschmann
(University of Pennsylvania)

CP Newsletter: Editorial or Ideology?

Having just read the introduction to the Comparative Politics newsletter, I found the lead editorial outlining the "myths" of DA-RT to be (presumably unintentionally) quite ideological. These are not “myths”: they are claims of how the DA-RT principles may be--and in some individual cases already have been--interpreted by reviewers and editors, thus resulting in unfair and inappropriate assessments of academic work deploying non-quantitative methods. There are many fine articles in this newsletter, and we can be grateful to the editors for providing the forum in which these issues can be aired. But the implication of the editorial—again, perhaps unintended--is to skew the conversation in a particular pro-DA-RT way. Isn't this exactly the kind of "bias" that DA-RT is supposed to guard against? Certainly, editors may have particular views—that’s what makes it an editorial, after all. But these should be based on an accurate reflection, and serious consideration, of the wide variety of views expressed on this issue—including some of the articles in that very same newsletter--rather than dismissing them as “myths.” That gesture moves the editorial from “considered judgment” to “ideology.”

by Nancy J. Hirschmann
University of Pennsylvania

(Response also posted at https://www.qualtd.net/viewtopic.php?f=10&t=52&sid=3ba21cf522dd27e7b9ef3....)

Response from Donald D. Searing
(University of North Carolina at Chapel Hill)

I think that the editor of World Politics has it about right. Hall is on target as well. We are talking about implementing a system that will have very undesirable unintended consequences. It will certainly discourage people, and especially young people, and particularly people in comparative politics, from doing ambitious qualitative-quantitative/mixed research. And it will discourage people from interviewing elites and others under real conditions of confidentiality (eg, "Turn it over") and from collecting complex, time-consuming interview and other qualitative data (eg, "Turn it all over in two years"). The model looks as though it is looking to micro economics whose model is physics. I think it is misguided. We are a methodologically rich diverse discipline and that is our strength. Are some colleagues hoping we can become a "real" science rather than be a social science (which uses natural science methods to study humanistic problems) by getting rid of anything that smells of the humanities? I spent a morning reading all of this newsletter and got so worked up that I'm posting about something I would ordinarily never post about. Thankfully the current editors of the APSR seem to be bending over backwards to be accommodating. But who knows what the next APSR editors will do? Talk about incentive structures: under this regime, graduate students and new assistants in comparative politics would be absolutely bonkers (until tenure or even promotion to full) to do anything other than analyze existing quantitative data sets because any of their competitors doing just that will have an enormous advantage. Transparency yes, but with these costs? How did we get to this impasse?

by Donald D. Searing
University of North Carolina at Chapel Hill