The Open Science Peer Review Oath

from the transparent,-reproducible-and-responsible dept

Open access is about making academic research more widely available, particularly when it is publicly funded. But there is a broader open science movement that seeks to make the entire scientific process -- from initial experiments to the final dissemination of results -- transparent, and thus reproducible. One crucial aspect of that complete process is peer review, whereby experts in a field provide advice about the quality of new research, either to editors prior to a paper being published in a journal, or more directly, by reviewing work publicly online. Recognizing the importance of this step for the integrity and validity of the scientific process, a group has drawn up what it calls the "Open Science Peer Review Oath":
We have formulated an oath that codifies the role of reviewers in helping to ensure that the science they review is sufficiently open and reproducible; it includes guidelines not just on how to review professionally, but also on how to support transparent, reproducible and responsible research, while optimising its societal impact and maximising its visibility.
The Oath's 17 components include commitments to act fairly and ethically, for example, the following:
While reviewing this manuscript:

i) I will sign my review in order to be able to have an open dialogue with you

ii) I will be honest at all times

v) I will not unduly delay the review process

vi) I will not scoop research that I had not planned to do before reading the manuscript

vii) I will be constructive in my criticism

x) I will try to assist in every way I ethically can to provide criticism and praise that is valid, relevant and cognisant of community norms
It also includes actions specifically designed to foster science that is truly open:
xi) I will encourage the application of any other open science best practices relevant to my field that would support transparency, reproducibility, re-use and integrity of your research

xiii) I will check that the data, software code and digital object identifiers are correct, and the models presented are archived, referenced, and accessible

xiv) I will comment on how well you have achieved transparency, in terms of materials and methodology, data and code access, versioning, algorithms, software parameters and standards, such that your experiments can be repeated independently

xv) I will encourage deposition with long-term unrestricted access to the data that underpin the published concept, towards transparency and re-use

xvi) I will encourage central long-term unrestricted access to any software code and support documentation that underpin the published concept, both for reproducibility of results and software availability
Although the framing of an "Oath" for open science peer review may sound rather over the top -- slightly pompous, even -- it rightly underlines the seriousness with which peer review ought to be conducted. It remains to be seen what kind of response it receives from the wider scientific community, and whether it becomes a fixed element of the open science movement.

Follow me @glynmoody on Twitter or identi.ca, and +glynmoody on Google+

Filed Under: knowledge, open access, open sciencie, peer review, sharing


Reader Comments

Subscribe: RSS

View by: Time | Thread


  • identicon
    Anonymous Coward, 14 Nov 2014 @ 5:40am

    I appreciate the idea behind this, but I don't think this will actually prevent incidents like that one scientist who "proved" that wine was amazingly healthy for you (and happened to own several vineyards). People who twist the truth to suit their own desires wouldn't think twice about breaking an oath.

    reply to this | link to this | view in chronology ]

  • icon
    tqk (profile), 14 Nov 2014 @ 7:48am

    "cognisant of community norms "

    What does this--^ mean, and what has it got to do with independent research? I don't want to see researchers hamstrung by tradition. They're *supposed* to be exploring the boundaries of the known and unknown, not giving fealty to ideas of common decency and accepted institutionalism and the "status quo."

    reply to this | link to this | view in chronology ]

  • icon
    Scott Yates (profile), 14 Nov 2014 @ 10:33am

    What about negative results?

    I know it is probably outside of the scope of this kind of oath, but I think there should be more work done to encourage publication of results that did NOT show what the researchers/scientists set out to. Negative results are being swept under the rug and ignored. This is a problem that really seems like it should be addressed.

    reply to this | link to this | view in chronology ]

    • icon
      tqk (profile), 14 Nov 2014 @ 11:12am

      Re: What about negative results?

      I know it is probably outside of the scope of this kind of oath, but I think there should be more work done to encourage publication of results that did NOT show what the researchers/scientists set out to.

      I agree. Imagine:
      "In a recently completed, peer reviewed study, backed by four other studies replicating the experiment, it has been shown that violence on television, in books, and in video games shows no correlation with real world violence whatsoever. A related study shows the same is true for sexually related activities. It turns out that fiction is fiction and the reality of its observers is unaffected by fictional representations. This result is contrary to the researchers initial hypothesis and they continue to study their data in an effort to determine where they went wrong."

      In a perfect world.

      reply to this | link to this | view in chronology ]

  • icon
    toyotabedzrock (profile), 15 Nov 2014 @ 7:33pm

    This will never work. You cannot have people who have not earned their reputation reviewing things.
    The majority of society is not qualified to tell when a politician is lying.
    Further there is too much to be gained in cheating this system and this lowers the price for them to do so.

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Defender, 18 Nov 2014 @ 9:15am

    Rule #1 is a bad idea

    De-anonymizing reviews is a bad idea. A review is either valid or it's not, but the reputation/identity of the reviewer should not be a factor in that determination. If reviewers are not anonymous, more deference will be paid to reviews by respected, elder members of the profession, and less to reviews by junior, younger members. In addition, de-anonymizing reviewers reduces the pool of potential reviewers or calls into question the quality of the review since, for example, a junior scientist is less likely to negatively review the work of someone she knows will be on a board making recommendations for future grant awards.

    An editorial board can mitigate many of the bad effects of an anonymous peer review system, but can do much less about the deleterious effects of a non-anonymous system.

    reply to this | link to this | view in chronology ]


Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here



Subscribe to the Techdirt Daily newsletter




Comment Options:

  • Use markdown. Use plain text.
  • Remember name/email/url (set a cookie)

Close

Add A Reply

Have a Techdirt Account? Sign in now. Want one? Register here



Subscribe to the Techdirt Daily newsletter




Comment Options:

  • Use markdown. Use plain text.
  • Remember name/email/url (set a cookie)

Follow Techdirt
Techdirt Gear
Show Now: Takedown
Advertisement
Report this ad  |  Hide Techdirt ads
Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Chat
Advertisement
Report this ad  |  Hide Techdirt ads
Recent Stories
Advertisement
Report this ad  |  Hide Techdirt ads

Close

Email This

This feature is only available to registered users. Register or sign in to use it.