First Experience Reviewing a Paper

To help out a lab-mate, I agreed to review a conference paper last weekend. I was a bit nervous since I had never done this before, nor did I believe I was terribly qualified. Still, it was a good opportunity to change these facts, so I took it on.

My lab-mate gave me a really good article about refereeing to read. I found this paper refreshing, mostly because of its high quality writing (why does that seem so hard to find these days?). It was incredibly insightful, giving really good advice for how to review a paper, and at the same time, how to write a good paper yourself. It includes many questions to ask yourself about the paper you are reading (or writing). I highly recommend it for everyone in academia.

Armed with my newfound knowledge, I tackled the conference paper I was assigned. Fortunately, it was at just the right level, so I was able to fully understand what the authors were saying. I felt they had a good result, but that there were some serious issues with how they presented it.

The short summary is that they used up some of their four page limit early on in the paper for explaining some things that I don't think were necessary to the rest of the paper. Then they didn't have the space to elaborate on the key point of their method, which was reduced to a single sentence. Furthermore, they did not compare their results with the previous algorithm, even though they claimed theirs performed much faster, and handled more complex input.

I wasn't sure if this situation - that is, a paper that contained a good result but that needed some serious revisions in order to present this fact - warranted a rejection or not. I was told that I don't need to be too harsh, given that I was reviewing for a conference, not a journal. So the paper received a 'borderline' grade.

I hope that some of my suggested revisions will be made before the conference, because it would be such a shame to have a mediocre paper when it could have been great. (Or, another way to look at is that I hope that my faith in the quality of conference papers can be sustained by seeing this paper revised before acceptance.)