Like most students, I've been asked to review papers in my area (and a few that were pretty far outside it), and I always try to do a good job - this is definitely a golden-rule situation. If I don't take it seriously, I am absolutely convinced that karma will get me in the end, denying a crucial publication that could have pushed me over the edge to tenure.

I've also been lucky enough to have the fascinating experience of helping out with the Program Committee of a major conference, something students don't usually get to do. It is the kind of experience that really gives you perspective, and it's harder to get upset about disappointing results since then. Important decisions often come down to the quality of the reviewers and practical constraints - for instance, you may have space for 12 papers in the area, and you might be looking at a paper has two 'strong accept' reviews and one 'weak', but is pretty good. It seems like a borderline paper that might get in, right? But there are probably 20 others that got three 'strong accept' reviews - this paper has no practical chance unless someone champions it and the 'weak accept' reviewer wasn't very convincing.

The point of that little anecdote was that every review counts, even student reviews, and good reviews make the program committee's job a lot easier.

Off the top of my head, here are a few rules of thumb to reviewing:

Read the whole thing. It's only fair.
No matter what the questions on the review form say, always include a summary in your own words of the main point and contribution. This is really helpful to the author to put your other comments in perspective.
Don't be a wimp. If you mean 'reject', say so. If you use 'weak accept', explain why.
If there's space to add comments to the program committee, use it. Especially if you could be convinced to change your opinion of the paper. That can be useful if another reviewer had a very different opinion and the committee needs to reconcile them.
You're reviewing for a specific venue - if the paper is good, but you can think of a better place for it, say that and name that place - maybe the authors won't have thought of it, and at least it'll soften the blow a bit if it doesn't make it.
Take the time to scan the references - if they cite their own or similar work, check it out. This could be the only way you can answer the novelty question - how else will you know if they wrote the same paper six months ago and only added one result just to get to go on a nice trip?
Be honest about your expertise - it can help with the decisions, and it's hard to tell if you were dismissive because the paper was crap or because you don't understand its importance.

Does anyone else have a good tip for reviewing? Let me know in the comments.

Comments