Thursday, August 15, 2013

On publication lag


One potential negative of being an academic - especially for someone as impatient as I am - is the time lag of publication. It can easily take two years from the first submission of a manuscript to when that manuscript appears in print (in some high-profile journals and faster-paced fields the whole process can be shorter, but I'll focus on psychology here). What I want to argue is that, although publication lag has a whole host of negative consequences, it can nevertheless be a pathway to better work.

As a way to shortcut the long journal submission process, I've made a habit of submitting much of my research to the Cognitive Science Society. This is a great way to get findings out quickly: short papers are due in February, reviewed by April, and presented in July. The downside is that publications are not archival. Unlike in Computer Science - where conference publications are the standard - for purposes of jobs, promotion, etc., psychology papers must be published in a peer-reviewed journal. So I often write up some more complete version of my CogSci papers as substantially longer journal articles.

I recently tried to estimate the lag in this process. In 2007, 2008, and 2009 I submitted a number of papers to CogSci:
In other words, the lag was typically 3 - 5 years between the conference and the journal publication date. I feel exhausted even thinking about a lag of this magnitude: Ideas I was excited about last February will most likely see print in 2018.

By slowing down the broader dissemination of ideas, this lag clearly has negative consequences. Although conference proceedings are citable, journal papers stand a much better chance of having a long-term impact. A nicely typeset article in a good journal suggests solid research; when the article is published, there are often press-releases and content alerts that go out; and journal papers are indexed in PubMed and other catalogs as part of the archival scholarly literature. Yet that journal article is often several years out of date before we read it. 

If I look at the list of articles above none of them were published, or even submitted, with only minor changes from their CogSci versions. Instead, I made substantial revisions and additions prior to submitting them for the first time. Sometimes I replaced entire experiments - in some cases all of the experiments - because I had learned how to design them better or had created a better stimulus set. I was and still am happy with the initial CogSci papers. But taking the time to write them up, get reviews, prepare a talk, and present gave me space to become dissatisfied. It gave me time to up my standards and to think that I could do better. 

Peer review plays a part in this process, but the feedback I receive is not always critical in my revisions. In some case I make changes to please reviewers. But my most successful revisions are the ones in which I find a shared concern with the reviewers: a flaw that I recognize and that I am unsatisfied with. Then when I fix this flaw to my own satisfaction, reviewers are also satisfied. It takes time for me to get this kind of perspective.

That's why I think the lag itself is valuable, even independent of feedback from the review process. The slow speed of scientific publication may actually be a form of being tied to the mast. As frustrating as it is to wait 2 - 3 months for reviews, the process actually enforces the dictum of setting a draft aside, a practice that is endorsed by writing coaches from Stephen King to the Harvard Writing Center.* And without those enforced breaks, I doubt that I would have the discipline to keep from pressing "publish" and sending my (perhaps interesting but often half-baked) work out into the world.

A lot  has been written about changing publication standards for psychology and for science more generally. I especially like the Scientific Utopia pieces of Nosek and colleagues that describe ways that digital communication can help with disseminating scientific knowledge. But as much as I hate to say it, I wonder whether the molasses-slow timeline of scientific publication doesn't sometimes lead to better thought-out, higher-quality papers...


---
* Of course, there are some parts of the publication process that don't provide a benefit, e.g. the lag from proofs until the journal actually decides to print the darn thing. This lag could be eliminated if we gave up on paper journals. While many journals now use e-pub before print, my experience is that this practice leads to huge messes in Google Scholar and elsewhere when the same paper is cited to two different years.

2 comments:

  1. What we'd like to know is what the optimal amount of slow-down is.

    BTW if you only wait 2-3 months to get reviews back, I need to start getting your reviewers! 2-3 months has been the lower bound for me, and I've heard of people waiting as much as 12 months.

    ReplyDelete
  2. Absolutely - the optimal slowdown has got to vary by person and by idea. For me it needs to be long enough to engage in a new project and get my mind off the thing so that I can come at it fresh... but I would say that it's rare for me to need much more than a few months.

    As for the review process, I think that 2-3 months is somewhat reasonable. Sometimes I feel like I do really need a month or six weeks to chew over a paper (especially if I had other stuff to do in the mean time) - and then there's getting the reviewers and writing a decision letter. Anything more than that seems excessive, though.

    ReplyDelete