Bereits im Jahr 2008 wetterte der langjährige Ex-Editor des British Medical Journal, Richard Smith, in Lab Times gegen das traditionelle Prepublication Peer Review-System. Jetzt legt er in einem Editorial im Journal of Medical Screening (vol. 18(3):113-14) noch einen drauf:
The evidence, as opposed to the opinion, on prepublication peer review shows that its effectiveness has not been demonstrated and that it is slow, expensive, largely a lottery, poor at spotting error, biased, anti-innovatory (as perhaps in this case), prone to abuse, and unable to detect fraud. The global cost of peer review is $1.9 billion, and it is a faith-based rather than evidence-based process, which is hugely ironic when it is at the heart of science.
My conclusion is that we should scrap prepublication peer review and concentrate on postpublication peer review, which has always been the ‘real’ peer review in that it decides whether a study matters or not. By postpublication peer review I do not mean the few published comments made on papers, but rather the whole ‘market of ideas,’ which has many participants and processes and moves like an economic market to determine the value of a paper. Prepublication peer review simply obstructs this process.
Interessanterweise haben die zwei US-Kognitionsforscher Dwight Kravitz und Chris Baker gerade einen sehr passenden Aufsatz in Frontiers of Computational Neuroscience veröffentlicht, in dem sie vorschlagen, wie eine Kombination aus Pre- und Postpublication Peer Review tatsächlich funktionieren könnte. Der Titel: Toward a new model of scientific publishing: discussion and a proposal.
Bevor sie darin ihre Änderungsvorschläge ausbreiten, analysieren sie indes zunächst einmal anhand eines Sets von Artikeln aus sechs ausgewählten Zeitschriften (darunter etwa Science und Nature) den Status quo des Peer Review-Systems. Allein diese Analyse ist schon sehr lesenswert, wie etwa die folgenden zwei Zitate belegen:
Scientific papers are published through a legacy system that was not designed to meet the needs of contemporary scientists, the demands of modern publishing, or to take advantage of current technology. The system is largely carried forward from one designed for publishers and scientists in 1665.
In total, each paper was under review for an average of 122 days but with a minimum of 31 days and a maximum of 321. The average time between the first submission and acceptance, including time for revisions by the authors was 221 days (range: 31–533). This uncertainty in time makes it difficult to schedule and predict the outcome of large research projects. For example, it is difficult to be certain whether a novel result will be published before a competitor’s even it were submitted first, or to know when follow up studies can be published. It also makes it difficult for junior researchers to plan their careers, as job applications and tenure are dependent on having published papers.
Die Autoren schließen aus der Analyse:
Luckily, these deficiencies are structural and do not arise because of evil Authors, Reviewer, or Editors. Rather, they are largely a symptom of the legacy system of scientific publishing, which grew from a constraint on the amount of physical space available in journals. The advent of the Internet eliminates the need for physical copies of journals and with it any real space restrictions. In fact, none of the researchers in our lab had read a physical copy of a journal in the past year that was not sent to them for free. Without the space constraint there is no need to deny publication for any but the most egregiously unscientific of papers. In fact, we argue that simply guaranteeing publication for any scientifically valid empirical manuscript attenuates all of the intangible and quantifiable costs described above. Functionally, publication is already guaranteed, it is simply accomplished through a very inefficient system. 98.2% of all papers that enter the revision loop are published at that same journal and few papers are abandoned over the course of the journal loop.
Und dann machen sie sich daran, ihr Modell aus garantierter Online-Publikation sowie einer Mischung aus Vorab-Peer Review und Forums-basiertem Postpublikations-Review ausführlich im Detail zu entwickeln. Doch das schaut man sich tatsächlich besser im Original an….
Schlagworte: Gutachten, Lab Times, Markt, Online-Publikation, Open Access, Paper, Peer Review, Studie
Fully agree with R. Smith. Die wissenschaftliche Community und die Zeit entscheiden ja sowieso über inhaltlichen Bestand eines Manuskripts. Es geht eben dennoch bei den vielen hochrangigen Journals um Macht und Kontrolle über Information und hM (herrschende Meinung), auch im Hinblick auf die sehr interessanten Verquickungen der Editoren und Wissenschaftler. Aber ob eine „Liberalisierung des Marktes“ das gewünschte Ergebnis liefert?
Das oben angesprochene Modell (Kravitz u. Baker) könnte eine gute Lösung sein, wird sich vermutlich aber in naher Zukunft nicht durchsetzen.