So this is a thing…
|Was This Study Even Peer-Reviewed?
How journalists should write about unpublished scientific research.
Amid the nightmare of the past three weeks, with black men being shot by cops, and then black men shooting cops, the New York Times ran a surprising front-page headline…
This raises some interesting points regarding academic publishing. It is generally true that research has been more professionally vetted by the time it is published in a peer-reviewed journal, but it is also true that most research has been presented in seminars, etc. before reaching working paper status, and these environments provide opportunities for vetting, feedback, objection, and so on as well. From a logistical perspective, it’s hard to ignore how long the academic publishing timeline is (often measured in years not months), which means that a lot of research would be somewhat stale before it achieved peer-reviewed status. (Personally, I feel like fixing this process needs to be made a priority.)
On another note, however, I think a number of the criticisms of the study are brought up as caveats in the paper itself, so it’s not particularly fair to write as though the author has tried to hide all weaknesses of the data. Working papers are generally more or less at the stage where they are submitted to journals, so they tend to anticipate reviewer concerns and objections.