Also as an editor, I have asked various people to be reviewers for articles, and what I've received back from said reviewer I have had to significantly tone down the language in fact rewrite the review, because the reviewer had been so unnecessarily harsh on the author. Being hyper critical isn't the same as offering a critique.
So with that said here is my short guide to becoming a good reviewer/reader for a professional journal or academic publication.
This is not about you! Being a reviewer is not a platform to show the world how great a scholar you are. It is not an opportunity to showcase your vast knowledge of the subject. It is not a platform from which you can demonstrate to all and sundry how much better you are than the author of the article or book that you are reviewing. So the first rule of becoming a good review were is to check your ego at the door.
A good reviewer puts scholarship ahead of their own individual needs. This means that when approaching an article or book for review, it is about the contents of the article or book. How it is written, how it is constructed, does it add to the field, does it handle evidence well, is there secondary literature, and other types of important questions that we can ask about an article.
To be a good reviewer, then, first means a very important shift: it's about the scholarship, not about your scholarship.
It's a teaching moment. Reviewing an article or book is not unlike assessing student work, i.e. grading. This is an opportunity for the reviewer to teach and help the one reviewed, or the very least collaborate if the writer is someone more senior. Suggest revisions, suggest bibliography (that is appropriate!): always approach this with the goal of building a better paper/book rather than seeing how much you can tear down.
When suggesting revisions make certain that they are good revisions. I had a recent reviewer tell me that an Old English word did not say what I said it did, yet every lexicon and glossary of Old English (Bosworth-Toller, DOE, etc...even the Dict of ME) agreed with me. Before the reviewer suggests revisions, make certain that you the reviewer are actually correct!
When suggesting bibliography, also make certain that you know what you are talking about. I had a reviewer reject an article for publication. Among the reviewer's criticisms was that I had not quoted Big Name's Very Important Book nearly enough. But, Big Name in that Very Important Book mentioned the issue my paper was addressing once, made one point on that issue and talked about it for about a page. I cited that page, examined his point and explanation and took it apart. It was unnecessary to cite Big Name in that book any further because he was dealing with other things not germane to my own paper. The same reviewer also said that my paper's argument had been made by Other Big Name in Very Big Article 30 years previously, except that Other Big Name in Very Big Article never mentioned the texts I was working on nor the chronology or causation. In short, do not do this! If you the reviewer refers the author to bibliography, make certain the bibliography actually fits.
Along the same lines, avoid suggesting bibliography just to suggest bibliography. Just because an article mentions Unferth but does not in fact mean the author needs to cite every article that has examined Unferth, Reviewers should always ask themselves the question whether the bibliography they want to suggest is actually necessary to the paper's argument and examination, or is it just a case of "my bib is bigger than your bib."
If you accept an assignment to review, make sure you actually can. I have committed this sin far too many times. A recent book review I am ashamed to admit took me a year and a half to do....life kept getting in the way. On the other hand, as an editor I've had to chase down reviewers all too often too.
We all have a tendency to view what we can do with optimism...thinking we can do more than we really can. And life has a way of throwing us curveballs...and I for one am rarely prepared for the sudden curve ball of having so much to do suddenly that I had not planned on.
All that said means have a care when volunteering. Don't be a Larry.
Do not take the opportunity of double blind or anonymous reviewing as an opportunity to be snarky and rude. Like the now ancient cartoon, "On the Internet, no one knows you're a dog."
Write reviews that you would like to receive. I suppose one could say that upon this rule hang all the others.
The worst or best rejection I have ever received was "Great paper" accompanied by a rejection. This still puzzles me: why the rejection? Do both the editors and the author a favor and be specific on why suggesting rejection or revisions. If you don't, then the editor has to deal with distraught author and that eats up editor's time.
If the submission needs cleaning up, (typos, grammar errors, citation errors) simply say so with a few examples of the problem rather than a list of everything. This makes everyone's life more pleasant.
Along with Rule V above, do not undertake a review that you have no or little expertise in. If one has to study to be able to do a good review, then really the reviewer is doing a disservice rather than a service. So if asked to do a review, be honest if it is not within your expertise areas.
This should go without saying, but I'll say it anyhow. Um, the reviewer should actually read the submission....more than once is desirable. Do not turn in a review that is based on having not read or only partially read or merely skimmed the piece.
Stick to the major issues! It is easy to discuss at length a minor point in the submission, but if it isn't a major issue, touch on it lightly or not at all. The devil is not in the details on this one.
Keep in mind that doing reviews has a dual audience and dual purpose. First, the review and reviewer is collaborating with the editor(s) and helping the editor make a decision about the manuscript. Second, the review is to aid the author in improving the manuscript.
As a last comment, this is about you. No, I'm not changing my mind about Rule I! What I mean is that the best reviewers are those who approach doing a review as a learning opportunity for him/herself as much as a service opportunity to the editor/journal/press/field and a teaching opportunity for the author. But for yourself as a reviewer, be ready to learn.
Additional thoughts are welcome in the comments!!