Should Peer Reviewers Be Paid for their Work?

The following was originally posted to Talking Brains.

We'd all like our papers reviewed in a timely fashion by competent, thoughtful, and constructive reviewers. Needless to say, this doesn't always happen and it is easy to blame editors or journal policies for sluggish or unenlightened feedback on our journal submissions. While I don't like the policies of some journals/editors -- e.g., the policy of some journals to reject if any of the reviews are even just a nudge less than positive -- I can tell you as an editor, that the problem is most often the fault of the reviewers, i.e., you and me, the very people who complain about the peer review process.

Reviewing a paper is tedious, time-consuming, and generally thankless. Many of us get several review requests a month (or a week!). If you tend to review quickly and/or competently, you are more likely to get targeted for further review requests. So what do we do? We decline to review. As an author submitting a paper for review, we often think that the paper is "in review" basically from the moment of its submission. Sometimes it is possible to find reviewers within a day or two. Sometimes though, it can take weeks to find reviewers, especially if prospective reviewers don't respond to the original request for several days, or at all. For some papers I've edited, I've had to contact up to a dozen people before finally getting two reviewers to agree. Once in a while, I'd get a review back from one reviewer before I even find a second reviewer to agree to review the paper!

Then there is the delay in actually getting the reviews back. Most people are punctual, but certainly not all. If it takes a few iterations of review invitations to find reviewers and then one of the reviews is delayed, this can add up to substantial delays in response to the author even for journals that strive for rapid turn arounds.

Who ends up reviewing your paper? Well, in many cases it is junior scientists. Senior folks in the field tend to be more crunched for time and receive far more review requests. The people who end up agreeing to review (often via referrals from the first choice scientists) are more junior who have more time and more to gain (you get to add it to your CV!). Many junior scientists provide excellent reviews, but lack of experience and the broader perspective that comes with it can lead to weak reviews in various ways. This is the source of at least some of those slightly misinformed reviews we've all received.

So what can we do? For starters, I think it helps to realize that *we* are part of the problem. Next time you get frustrated with journal turnaround time, think twice about declining that next review invitation. A simple thing we all can do is respond quickly to review requests, even if it is to decline, and provide thoughtful alternative suggestions for other reviewers.

Some journals have instituted a reward system for reviewers, for example, if your review is on time you get entered into a lottery to win a prize. I don't find this particularly motivating though. Frontiers journals have an interesting policy: for every paper that comes in, every review editor (the editorial board, which is generally large for these journals) gets an invitation to review the paper. This seems to work because every time I click my response to these review requests, I get a message saying that the required number of reviewers has been found. Maybe this approach would work more generally. It is annoying to get this constant flood of invitations though, and pretty soon you realize it is easier to say 'no thanks' when you know there is a large pool of reviewers who received the request.

So what about paying reviewers? Would you be more willing to review a paper if you were paid $50 or $100 (if it was submitted on time of course!)? NIH pays grant reviewers to help offset the time commitment and effort. Why shouldn't journals pay too? Maybe they can't afford to, but they could impose a submission fee that would cover the cost of reviewer payment. Would you be willing to pay to have your paper reviewed?

What are your thoughts?