With Great Power Comes Great Responsibility

Author:

Omer Levy (University of Washington)

PC Chair Note: Omer is an outstanding reviewer and author, one of my favorite rising stars. -Heng

Writing reviews is a time-consuming task, and it can be tempting to get it over quickly or view it as one of the thankless chores of academia. However, it’s also a rare privilege; rather than focusing exclusively on our own work, we have the opportunity to evaluate and influence the work of others in our  community. Our opinion contributes to the decision of which body of knowledge is disseminated among our peers. With great power comes great responsibility – so what is the right way to review a paper?

Different reviewers tend to be more sensitive to different aspects of a paper. For example, I find myself placing more emphasis on ensuring a paper’s soundness and reproducibility, while another reviewer may focus on the same paper’s originality. Having a diverse set of independent reviewers is a good thing, because frankly, there isn’t one single formula for constructing a high quality review. Having said that, there are a few points that I believe every reviewer should take into account.

Treat the paper with respect. The authors are your peers, and they have probably spent the better half of the past few months working on this paper. Read the paper carefully and go the extra mile in trying to understand their contribution, even if you end up spending an extra hour on it. There’s nothing quite as frustrating as having your paper rejected because of a lazy review.

Be critical. Although we aspire to perfection, every paper has room for improvement. Are all of the arguments supported by sufficient evidence? Is the model’s description clear and reproducible? Is the experimental setup flawless? Being critical goes hand in hand with treating the paper with respect, because only after we have done our best to understand the paper can we really be confident about our opinion of it.

Write useful reviews. Being the devil’s advocate is only half the job. For each flaw that you find, try to explain to the authors why it is an important issue and, more importantly, how to fix it. For example, rather than just saying “you didn’t cite X,” describe in detail how the work relates to X, and what kind of comparative discussion or experiment you expect to see. If you find yourself struggling with explaining why and how a problem should be addressed, then perhaps it’s not such a big deal after all.

Whether you are a young grad student or a senior professor, reviewing is a unique opportunity to influence the community. A well thought-out review can have a very positive effect on a paper’s quality, while an offhand review might block good ideas from propagating or let erroneous results become canon. What kind of review are you going to write?

Advertisement