[Initial posting on this topic is reproduced below; the response follows it.]
From: "Gerry Mckiernan" <gerrymck_at_iastate.edu>
Little Evidence for Effectiveness of Scientific Peer Review
A Most Revealing (and Perhaps Disturbing) Report on Effectiveness of
Scientific Peer Review.
[Thanks, Ben Toth, NHS Information Authority (UK) for informing me of this
major report!!!]
The report focuses on biomedical journals. I'd be interested in Any and
All similar studies for *other* disciplines.
Gerry McKiernan
Effective Librarian
Iowa State University
Ames IA 50011
gerrymck_at_iastate.edu
Little Evidence for Effectiveness of Scientific Peer Review by Caroline
White / BMJ 2003;326:241 ( 1 February )
[http://bmj.com/cgi/content/full/326/7383/241/a ]
DateLine: London
Despite its widespread use and costs, little hard evidence exists that
peer review improves the quality of published biomedical research,
concludes a systematic review from the international Cochrane Collaboration.
[ http://bmj.com/cgi/reprint/326/7383/241/a.pdf ]
Yet the system, which has been used for at least 200 years, has only
recently come under scrutiny, with its assumptions about fairness and
objectivity rarely tested, say the review authors. With few exceptions,
journal editors-and clinicians-around the world continue to see it as the
hallmark of serious scientific endeavour. Published last week, the review
is the third in a series from the Cochrane Collaboration Methods Group.
...
Only the latter escapes a drubbing, with the reviewers concluding that
technical editing does improve the readability, accuracy, and overall
quality of published research.The Cochrane reviewers based their findings
on 21 studies of the peer review process from an original trawl of only 135.
[snip]
On the basis of the current evidence, "the practice of peer review is based
on faith in its effects, rather than on facts," state the authors, who call
forlarge, government funded research programmes to test the effectiveness
of the system and investigate possible alternatives.
"As the information revolution gathers pace, an empirically proven method
of quality assurance is of paramount importance," they contend.
Professor Tom Jefferson, who led the Cochrane review, suggested that
further research might prove that peer review, or an evolved form of it,
worked. At the very least, it needed to be more open and accountable. But
he said that there had never even been any consensuson its aims and that it
would be more appropriate to refer to it as "competitive review."
[snip]
REPORT FULL-TEXT AVAILABLE
Editorial peer-review for improving the quality of reports of biomedical
studies
[ http://www.update-software.com/Cochrane/MR000016.pdf ]
Jefferson TO, Alderson P, Davidoff F, Wager E
This is a reprint of a Cochrane methodology review, prepared and maintained
by the Cochrane Collaboration and published in The Cochrane Library 2003,
Issue 1
=============================================================================================
(1)
From: John Rutledge <jbr_at_email.unc.edu>
If you're interested in criticism of the peer review process, you might
want to take a look at H.-D. Daniel's _Guardians of
Science: Fairness and Reliability of Peer Review_. (Weinheim: VCH, 1993;
ISBN: 3-527-29041-9.) (Reviewed by me in _Collection Management_ 19/1-2,
1994, 171-174.)
A few points:
1) Peer review has not been universally adopted. _Angewandte Chemie_
didn't start using peer review until 1982.
2) If reviewers disagree, what does an editor do? Psychological and
behavioral factors influence the peer review process.
3) A significant number of reviewers can identify "blinded" articles (but
fewer can than believe they can) when they receive
them for assessment.
4) A high percentage of articles rejected by Journal A are later submitted
to and published by Journal B.
5) Journals editors are able to predict accurately which articles will
later be frequently cited even without a peer review
process.
John Rutledge
* John B. Rutledge *
* Bibliographer, West European Resources *
+ Acting Head, Collection Development Department
* Davis Library CB# 3198 *
* University of North Carolina, Chapel Hill *
* North Carolina 27514-8890 *
* jbr_at_email.unc.edu *
* 919-962-1095 *
Received on Tue May 13 2003 - 01:53:07 EDT