Paper Reviews In Class

Our Open Source Engineering class is a little different from most here at Carleton's School of Computer Science. The professor wants to give students an opportunity to practice their communication skills not only via the standard in-class presentation, but also with many group discussions on topics related to open source. The most interesting part of the course, however, is the conference-style reviewing we do of our own papers.

Our project this semester was to design a fingerprint format for open source software. These fingerprints need to represent a JAR file well enough to be compared with fingerprints from other JAR files, yet be as compact as possible. Such fingerprints could then be created for common open source projects, and used to detect inappropriate inclusion into other software.

We had to finish our implementations a couple of weeks ago, and then write a conference-style paper about them for the following week. In last night's class, we reviewed three papers (with more to be looked at in the following weeks). One student acted as a moderator, and another as a summarizer. A third student took notes. The moderator had the paper's authors read one paragraph after the summarizer introduced it, then asked for positive comments on the structure and format of the paper. This was followed by negative comments on structure, and finally positive and negative comments on content. The authors were forbidden to speak during the comments, since in a real review they wouldn't even be present.

I was really impressed with how well this process went. We were very good at pointing out the good things in the papers, and provided insightful suggestions for improvement. I honestly didn't expect this level of quality. The whole idea of paper reviewing will not only result in much better papers at the end of the term, but give a good taste of the conference world. Since many of the students in the class are at the undergraduate or Masters level, it could even lead to a better chance of success for their first real paper submissions.

I would definitely recommend this kind of activity for any grad course, though I might also include a bit of an introduction on how to effectively read research papers; I noticed that many of the students' papers did not include a sufficient background section.