by Christopher Manning.
In October 2017, after much discussion and surveying, the Association for Computational Linguistics (ACL) adopted new policies on submission, review, and citation. A key change was to disallow conference submission of papers that have been posted or updated as non-anonymous preprints during a period from one month before the submission deadline until after results are announced.
I have seen several online discussions mainly critical of this preprint policy, variously describing it with terms like “idiotic” and “ineffective”. I think the ACL has failed to sufficiently explain to people the thinking and discussion underlying this aspect of the policy, so here is my attempt to do so.
Speeding up the rate of scientific progress by fast dissemination of results is a good thing. Double-blind reviewing to prevent a strong bias in favor of the already privileged is a good thing. These two good things conflict. The ACL policy is an attempt at a compromise between them.
As a compromise, it mandates some delay in (non-anonymous) dissemination and it only improves rather than attempting to make watertight the integrity of double-blind review. But I think it is a good compromise, which seems to be working as intended fairly well.
Maintaining author anonymity was never an absolute: people always gave talks on their work and sent it to colleagues. The new ACL policy in no way attempts to restrict these traditional forms of small-scale manuscript sharing, but attempts to distinguish them from public dissemination by the author(s) to the whole world, aided by the alerting services and recommendation engines that surround a preprint archive such as arXiv. Its main target was to prevent the rapidly increasing practice of posting non-anonymous preprints (often followed by social media/blog promotion) at approximately the same time as paper submission, which greatly undermined blind review.
If someone is not prepared to compromise at all on the fast dissemination of results in order to address issues of diversity, inclusivity, and (implicit) bias, then it really seems like these latter issues are not very important to them.
Overall, the ACL policy leans considerably towards favoring speeding science by allowing the highly valuable uses of preprints, such as posting new results when the next conference deadline is way off (e.g., in June, for our field) or when someone wants to get reactions from colleagues on an early version of some work, after which they will revise and develop it further. People can also post their rejected papers to avoid further delays in their dissemination, providing any next conference submission deadline that they intend to aim at is over one month off. Yet, nevertheless, all signs are that the policy is still quite effective at maintaining mostly double-blind reviewing.
The effectiveness of this aspect of the ACL policy mainly relies on two human failings: procrastination and forgetfulness. Everyone could finish their papers more than a month before the deadline but in practice very few people do. Some preprints and previously rejected papers will be broadly available but, since they must be from more than a month back, people are much less likely to remember their authors.
High-probability anonymity is all we ever had and it is sufficient to largely preserve the good of double-blind reviewing. The ACL preprint policy simply takes a step to keep that probability from being greatly eroded, while mainly allowing preprints for the sake of speeding science.
(Revised May 28, 2018, by removing a link and explicit reference to a preceding Twitter discussion. I apologize to Jimmy Lin for appearing to have attributed to him an opinion that he does not hold. It was not my intent to do that, but I regret having done so. It was a mistake for me to refer to the Twitter conversation in this blog post. Nevertheless, thanks to Emily Bender for her thought-provoking earlier discussion.)
NAACL HLT 2018 Chairs Blog 