The Open Science Peer Review Oath
from the transparent,-reproducible-and-responsible dept
Open access is about making academic research more widely available, particularly when it is publicly funded. But there is a broader open science movement that seeks to make the entire scientific process — from initial experiments to the final dissemination of results — transparent, and thus reproducible. One crucial aspect of that complete process is peer review, whereby experts in a field provide advice about the quality of new research, either to editors prior to a paper being published in a journal, or more directly, by reviewing work publicly online. Recognizing the importance of this step for the integrity and validity of the scientific process, a group has drawn up what it calls the “Open Science Peer Review Oath“:
We have formulated an oath that codifies the role of reviewers in helping to ensure that the science they review is sufficiently open and reproducible; it includes guidelines not just on how to review professionally, but also on how to support transparent, reproducible and responsible research, while optimising its societal impact and maximising its visibility.
The Oath’s 17 components include commitments to act fairly and ethically, for example, the following:
While reviewing this manuscript:
i) I will sign my review in order to be able to have an open dialogue with you
ii) I will be honest at all times
v) I will not unduly delay the review process
vi) I will not scoop research that I had not planned to do before reading the manuscript
vii) I will be constructive in my criticism
x) I will try to assist in every way I ethically can to provide criticism and praise that is valid, relevant and cognisant of community norms
It also includes actions specifically designed to foster science that is truly open:
xi) I will encourage the application of any other open science best practices relevant to my field that would support transparency, reproducibility, re-use and integrity of your research
xiii) I will check that the data, software code and digital object identifiers are correct, and the models presented are archived, referenced, and accessible
xiv) I will comment on how well you have achieved transparency, in terms of materials and methodology, data and code access, versioning, algorithms, software parameters and standards, such that your experiments can be repeated independently
xv) I will encourage deposition with long-term unrestricted access to the data that underpin the published concept, towards transparency and re-use
xvi) I will encourage central long-term unrestricted access to any software code and support documentation that underpin the published concept, both for reproducibility of results and software availability
Although the framing of an “Oath” for open science peer review may sound rather over the top — slightly pompous, even — it rightly underlines the seriousness with which peer review ought to be conducted. It remains to be seen what kind of response it receives from the wider scientific community, and whether it becomes a fixed element of the open science movement.
Follow me @glynmoody on Twitter or identi.ca, and +glynmoody on Google+
Filed Under: knowledge, open access, open sciencie, peer review, sharing
Comments on “The Open Science Peer Review Oath”
I appreciate the idea behind this, but I don’t think this will actually prevent incidents like that one scientist who “proved” that wine was amazingly healthy for you (and happened to own several vineyards). People who twist the truth to suit their own desires wouldn’t think twice about breaking an oath.
"cognisant of community norms "
What does this–^ mean, and what has it got to do with independent research? I don’t want to see researchers hamstrung by tradition. They’re *supposed* to be exploring the boundaries of the known and unknown, not giving fealty to ideas of common decency and accepted institutionalism and the “status quo.”
What about negative results?
I know it is probably outside of the scope of this kind of oath, but I think there should be more work done to encourage publication of results that did NOT show what the researchers/scientists set out to. Negative results are being swept under the rug and ignored. This is a problem that really seems like it should be addressed.
Re: What about negative results?
I agree. Imagine:
In a perfect world.
Re: Re: What about negative results?
I was not aware of this one and had to google it.
http://www.gamespot.com/articles/research-shows-no-link-between-video-game-violence/1100-6423385/
Thank you!
Maybe there IS hope for more of this.
This will never work. You cannot have people who have not earned their reputation reviewing things.
The majority of society is not qualified to tell when a politician is lying.
Further there is too much to be gained in cheating this system and this lowers the price for them to do so.
Rule #1 is a bad idea
De-anonymizing reviews is a bad idea. A review is either valid or it’s not, but the reputation/identity of the reviewer should not be a factor in that determination. If reviewers are not anonymous, more deference will be paid to reviews by respected, elder members of the profession, and less to reviews by junior, younger members. In addition, de-anonymizing reviewers reduces the pool of potential reviewers or calls into question the quality of the review since, for example, a junior scientist is less likely to negatively review the work of someone she knows will be on a board making recommendations for future grant awards.
An editorial board can mitigate many of the bad effects of an anonymous peer review system, but can do much less about the deleterious effects of a non-anonymous system.