I suppose most people know the story that the Beatles were turned down by several record companies before they finally got accepted by Parlophone: the rest is history; Richard Pirsig's remarkable book Zen and the Art of Motorcycle Maintenance was turned down by 121 (!) publishers before being accepted for publishing: yet it was an instant best-seller.
Doris Lessing is a well-known, best-selling author. In 1984 she decided to write a book under a pseudonym. The Diary of a Good Neighbour by Jane Somers was subsequently offered to Lessing's two main publishers (without telling them who had really written it, of course). They both turned the book down. Interestingly enough, the publisher who did finally accept the book was the same publisher who had accepted Lessing's first ever book. Of course the fun continued with the reviews of the book.
Lessing says: "I wanted to cheer up young writers ... by illustrating that certain attitudes and processes they have to submit to are mechanical, and have nothing to do with them personally, or with their kind or degree of talent".
It is shortly time to submit papers for CHI again, although there are dates throughout the year for different conferences.
I, like many of you reading this, have submitted articles and books for conferences and publication, and not all of them have been accepted. I have often read the reviewers' remarks with wonder, and the question "Have they actually read the paper?" has, alas, often enough passed my lips. I'm sure that I am not the only person who has sat through lectures at conferences and asked "How can they have accepted this, and rejected my paper?" I have even had the indignity of sitting through a plenary invited lecture that was in content the same as my, rejected, paper (although perhaps, on reflection, that was reason enough to reject it).
At INTERCHI '93 in Amsterdam, Professor Alan Newell, one of the two invited plenary speakers, reported during his talk that a paper that he had submitted to the conference had been reviewed and received remarks like "very important", with a review mark of 1 from some reviewers, and remarks like "of no importance" with a mark of 5 from others. The paper was rejected.
Now don't get me wrong. I'm not trying to say that all referees are bad. On the contrary, many are very good, and very often referees remarks can help to make a paper very much better; for goodness sake, I've been a referee myself often enough.
But the question remains: who referees the referees?
Sometimes at conferences I have had an unkind suspicion that papers are more likely to be accepted if the author is a famous name or comes from somewhere well known, or if the paper handles a minor subject, or a slight modification to current thinking, than a visionary statement from some unknown at a little-known institute.
Now, I might have attributed such feelings to sour grapes, that if I'd had all my papers accepted, I wouldn't feel them, except that a study1 has shown that in fact my suspicions are not ill-founded.
The study quotes a number of investigations into the process of reviewing papers, and reveals some pretty amazing occurrences. For example, in one case, 12 papers that had previously been accepted by a prestigious journal were cosmetically disguised and resubmitted. During reviewing only three were spotted, and of the rest, eight were rejected, and not because they added nothing new to the field, for instance, but for 'methodological shortcomings'.
Two other studies quoted showed that papers were significantly more likely to be accepted if they confirmed existing beliefs than if they contradicted them. Another study showed a bias against authors from unknown institutions by reviewers from well-known institutes.
Other studies showed that a paper is more likely to be accepted if it is written in complicated language rather than clear language (alas for us all), and that important problems do not get the priority that they would seem to deserve.
My main point is this: at conferences you see and hear many good presentations. Remember, especially if your paper was rejected, they are not necessarily all of them 'the best', and the rejected papers are not all 'the worst'.
Or, as Lessing said, certain attitudes and processes authors have to submit to are mechanical, and have nothing to do with them personally, or with their kind or degree of talent.
Reference
1 J. Scott Armstrong, Research on Scientific Journals: Implications for Editors and Authors, Journal of Forecasting, Vol 1, 83-104 (1982)
© Copyright Steven Pemberton, Amsterdam, 1994. All rights reserved.
First published in the SIGCHI Bulletin, July 1994