The Open Science Peer Review Oath

from the transparent,-reproducible-and-responsible dept

Open access is about making academic research more widely available, particularly when it is publicly funded. But there is a broader open science movement that seeks to make the entire scientific process — from initial experiments to the final dissemination of results — transparent, and thus reproducible. One crucial aspect of that complete process is peer review, whereby experts in a field provide advice about the quality of new research, either to editors prior to a paper being published in a journal, or more directly, by reviewing work publicly online. Recognizing the importance of this step for the integrity and validity of the scientific process, a group has drawn up what it calls the “Open Science Peer Review Oath“:

We have formulated an oath that codifies the role of reviewers in helping to ensure that the science they review is sufficiently open and reproducible; it includes guidelines not just on how to review professionally, but also on how to support transparent, reproducible and responsible research, while optimising its societal impact and maximising its visibility.

The Oath’s 17 components include commitments to act fairly and ethically, for example, the following:

While reviewing this manuscript:

i) I will sign my review in order to be able to have an open dialogue with you

ii) I will be honest at all times

v) I will not unduly delay the review process

vi) I will not scoop research that I had not planned to do before reading the manuscript

vii) I will be constructive in my criticism

x) I will try to assist in every way I ethically can to provide criticism and praise that is valid, relevant and cognisant of community norms

It also includes actions specifically designed to foster science that is truly open:

xi) I will encourage the application of any other open science best practices relevant to my field that would support transparency, reproducibility, re-use and integrity of your research

xiii) I will check that the data, software code and digital object identifiers are correct, and the models presented are archived, referenced, and accessible

xiv) I will comment on how well you have achieved transparency, in terms of materials and methodology, data and code access, versioning, algorithms, software parameters and standards, such that your experiments can be repeated independently

xv) I will encourage deposition with long-term unrestricted access to the data that underpin the published concept, towards transparency and re-use

xvi) I will encourage central long-term unrestricted access to any software code and support documentation that underpin the published concept, both for reproducibility of results and software availability

Although the framing of an “Oath” for open science peer review may sound rather over the top — slightly pompous, even — it rightly underlines the seriousness with which peer review ought to be conducted. It remains to be seen what kind of response it receives from the wider scientific community, and whether it becomes a fixed element of the open science movement.

Follow me @glynmoody on Twitter or, and +glynmoody on Google+

Filed Under: , , , ,

Rate this comment as insightful
Rate this comment as funny
You have rated this comment as insightful
You have rated this comment as funny
Flag this comment as abusive/trolling/spam
You have flagged this comment
The first word has already been claimed
The last word has already been claimed
Insightful Lightbulb icon Funny Laughing icon Abusive/trolling/spam Flag icon Insightful badge Lightbulb icon Funny badge Laughing icon Comments icon

Comments on “The Open Science Peer Review Oath”

Subscribe: RSS Leave a comment
Scott Yates (profile) says:

What about negative results?

I know it is probably outside of the scope of this kind of oath, but I think there should be more work done to encourage publication of results that did NOT show what the researchers/scientists set out to. Negative results are being swept under the rug and ignored. This is a problem that really seems like it should be addressed.

tqk (profile) says:

Re: What about negative results?

I know it is probably outside of the scope of this kind of oath, but I think there should be more work done to encourage publication of results that did NOT show what the researchers/scientists set out to.

I agree. Imagine:

“In a recently completed, peer reviewed study, backed by four other studies replicating the experiment, it has been shown that violence on television, in books, and in video games shows no correlation with real world violence whatsoever. A related study shows the same is true for sexually related activities. It turns out that fiction is fiction and the reality of its observers is unaffected by fictional representations. This result is contrary to the researchers initial hypothesis and they continue to study their data in an effort to determine where they went wrong.”

In a perfect world.

Anonymous Defender says:

Rule #1 is a bad idea

De-anonymizing reviews is a bad idea. A review is either valid or it’s not, but the reputation/identity of the reviewer should not be a factor in that determination. If reviewers are not anonymous, more deference will be paid to reviews by respected, elder members of the profession, and less to reviews by junior, younger members. In addition, de-anonymizing reviewers reduces the pool of potential reviewers or calls into question the quality of the review since, for example, a junior scientist is less likely to negatively review the work of someone she knows will be on a board making recommendations for future grant awards.

An editorial board can mitigate many of the bad effects of an anonymous peer review system, but can do much less about the deleterious effects of a non-anonymous system.

Add Your Comment

Your email address will not be published. Required fields are marked *

Have a Techdirt Account? Sign in now. Want one? Register here

Comment Options:

Make this the or (get credits or sign in to see balance) what's this?

What's this?

Techdirt community members with Techdirt Credits can spotlight a comment as either the "First Word" or "Last Word" on a particular comment thread. Credits can be purchased at the Techdirt Insider Shop »

Follow Techdirt

Techdirt Daily Newsletter

Techdirt Deals
Techdirt Insider Discord
The latest chatter on the Techdirt Insider Discord channel...