Leading Biomedical Funders Call For Open Peer Review Of Academic Research

from the nothing-to-hide dept

Techdirt has written many posts about open access -- the movement to make digital versions of academic research freely available to everyone. Open access is about how research is disseminated once it has been selected for publication. So far, there has been less emphasis on changing how academic work is selected in the first place, which is based on the time-honored approach of peer review. That is, papers submitted to journals are sent out to experts in the same or similar field, who are invited to comment on ways of improving the work, and on whether the research should be published. Traditionally, the process is shrouded in secrecy. The reviewers are generally anonymous, and the reports they make on the submissions are not made public. Now, however, the idea of making peer review more transparent as part of the general process of becoming more open is gaining increasing impetus.

A couple of weeks ago, representatives of two leading biomedical funders -- the UK Wellcome Trust and the Howard Hughes Medical Institute -- together with ASAPbio, a non-profit organization that encourages innovation in life-sciences publishing, wrote a commentary in Nature. In it, they called for "open review", which, they point out, encompasses two distinct forms of transparency:

'Open identities' means disclosing reviewers' names; 'open reports' (also called transparent reviews or published peer review) means publishing the content of reviews. Journals might offer one or the other, neither or both.

In a 2016 survey, 59% of 3,062 respondents were in favour of open reports. Only 31% favoured open identities, which they feared could cause reviewers to weaken their criticisms or could lead to retaliation from authors. Here, we advocate for open reports as the default and for open identities to be optional, not mandatory.

The authors of the commentary believe that there are a number of advantages to open reports:

The scientific community would learn from reviewers' and editors’ insights. Social scientists could collect data (for example, on biases among reviewers or the efficiency of error identification by reviewers) that might improve the process. Early-career researchers could learn by example. And the public would not be asked to place its faith in hidden assessments.

There are, of course risks. One concern mentioned is that published reviews might be used unfairly in subsequent evaluation of the authors for grants, jobs, awards or promotions. Another possibility is the 'weaponization' of reviewer reports:

Opponents of certain types of research (for example, on genetically modified organisms, climate change and vaccines) could take critical remarks in peer reviews out of context or mischaracterize disagreements to undermine public trust in the paper, the field or science as a whole.

Despite these and other concerns mentioned in the Nature commentary, an open letter published on the ASAPbio site lists dozens of major titles that have already instituted open reports, or promise to do so next year. As well as that indication that open reports are passing from concept to reality, it's worth bearing in mind that the UK Wellcome Trust and the Howard Hughes Medical Institute are major funders of biomedical research. It would be a relatively straightforward step for them to make the adoption of open reports a condition of receiving their grants -- something that would doubtless encourage uptake of the idea.

Follow me @glynmoody on Twitter or identi.ca, and +glynmoody on Google+


Reader Comments

Subscribe: RSS

View by: Time | Thread


  • icon
    Anonymous Anonymous Coward (profile), 5 Sep 2018 @ 6:46pm

    What is good for the goose isn't always good for the gander

    "Only 31% favoured open identities, which they feared could cause reviewers to weaken their criticisms or could lead to retaliation from authors."

    The answer to this concern is more reviews. At some point, thin skinned reviewers will, via their consistent obstinacy, be found out, and either not asked to review anything else or have their reviews discounted. I would think the point is, beyond finding actual error or educating up and comers, is to validate or discredit (based upon verifiable facts and not ideology) a papers findings. If there is error, an opportunity to correct and re-review is available. If there is ideological difference, then that ideological difference can be pointed out. The church says the world is flat, no discussion. Galileo on the other hand...

    "Opponents of certain types of research (for example, on genetically modified organisms, climate change and vaccines) could take critical remarks in peer reviews out of context or mischaracterize disagreements to undermine public trust in the paper, the field or science as a whole."

    Who might also be characterized as trolls. If they have some scientific, fact based, quantifiable arguments, let the present those. If not, then, like trolls here, they can be deemed unreliable, and not asked to review again, while the mischaracterized arguments could be refuted with facts. Can the president be marked a troll? Ahem!

    Personally, I have some problems with genetically modified organisms. My problem isn't that they are genetically modified, but that we don't actually know enough to deem them safe. Give some organism 50 or so years in a closed environment, then we might know. I don't have sufficient faith in companies who have dramatic economic interest in approving something as being beneficial to mankind to believe them. Look at roundup for example. Monsanto pushed the chemical for years and made a lot of money, but in the end.... How long did that take? Well it seems Monsanto introduced roundup in the 1970's and this court case finished in 2018. So forty years or so?

    More time is needed, and if the developers don't like those economic terms, well my health and your health is worth more than their profits. I am not saying that genetics cannot lead to better things. I am saying that the testing period is much too short and possibly not critical enough. Here we get back to the investors, or shareholders wanting more today and never thinking about tomorrow. One buys a stock, new product comes out, stock price goes up, investor sells, new product kills people, company gets sued, stock prices go down, but the people that funded the original research are not impacted because they sold their stock.

    Which makes a strong argument for government funding of research and not selling the patents, but licensing companies to produce a proven, and well tested product. The government gets their investment back (taxes go down, well, in my dreams, not likely the government won't find a way to spend that money on something not useful to us), and the company makes a profit. The researcher doesn't become rich, but if things are set up well, the deal could include some incentive to the researcher from the governments licensing fees. Just a thought.

    reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 6 Sep 2018 @ 4:45am

      Re: What is good for the goose isn't always good for the gander

      At some point, thin skinned reviewers will,

      Its not thin skinned reviewers who are a potential problem, but thin skinned authors who may have the power to affect a reviewers academic career.

      reply to this | link to this | view in chronology ]

    • identicon
      Christenson, 6 Sep 2018 @ 7:52am

      Re: What is good for the goose isn't always good for the gander

      Umm...given the corruption that funding sources cause, I think disclosing the funding sources of reviewers may do quite a bit to let people know about the motivations of an individual reviewer.

      reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 7 Sep 2018 @ 9:41am

      Re: What is good for the goose isn't always good for the gander

      Personally, I have some problems with genetically modified organisms. My problem isn't that they are genetically modified, but that we don't actually know enough to deem them safe.

      FYI, radiation breeding has been happening since the 1930s and a lot of foods are newer than people think. New types of apples show up all the time (though not via artificial radiation).

      The only reason I avoid GMO stuff is Monsanto's lawsuits against farmers by which they make it illegal to plant seeds.

      reply to this | link to this | view in chronology ]

  • identicon
    Christenson, 5 Sep 2018 @ 9:03pm

    Open Access Funding

    I think this story is incomplete without mentioning that there has been an escalation in the fight for open-access journals this week, with the major european funders saying that a condition of receiving their money will be publishing the results in full open-access journals as of, I think it was 2020.

    Anybody heard what's happening on the sci-hub legal front lately???

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 6 Sep 2018 @ 3:59am

    lolwut?

    reply to this | link to this | view in chronology ]

    • identicon
      Lawrence D’Oliveiro, 6 Sep 2018 @ 6:41pm

      Re: “whether a vaccine is safe and effective for its intended u”

      Well, the fact that you have so far avoided epidemics of measles, mumps, rubella, polio, smallpox etc would seem to be a pretty good criterion, don’t you think?

      reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 6 Sep 2018 @ 5:24am

    Manufacturers - no accountabiliby

    "Congress in 1987 had a number of options before it, including adopting an entirely different compensation scheme, as the Reagan administration was proposing; establishing different limitations on tort liability, including eliminating design defect liability, as pharmaceutical industry leaders were advocating;"

    https://supreme.justia.com/cases/federal/us/562/223/#F12





    (“[T]he liability provisions of the 1986 Act should be amended to assure that manufacturers will not be found liable in the tort system if they have fully complied with applicable government regulations. In particular, manufacturers should not face liability under a ‘design defect’ theory in cases where plaintiffs challenge the decisions of public health authorities and federal regulators that the licensed vaccines are the best available way to protect children from deadly diseases” (statement of Robert B. Johnson, President, Lederle Laboratories Division, American Cyanamid Co.)).

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 6 Sep 2018 @ 5:38am

    Faulty studies

    https://www.commondreams.org/views/2018/06/12/vaccines-and-liberal-mind

    Instead of demanding blue-ribbon safety science and encouraging honest, open and responsible debate on the science, too many online outlets are silencing critics and shutting down discussion on this key public health and civil rights issue.
    by Robert F. Kennedy Jr.

    Instead of the multi-year double-blind inert placebo studies—the gold standard of safety science...

    Equally shocking, FDA does not require vaccine manufacturers to measure proposed vaccines against true inert placebos, further obscuring researchers’ capacity ...
    .

    reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 6 Sep 2018 @ 9:53am

      Re: Faulty studies

      the issue has become politicized. I cannot be walked back now because the zealots on both sides cannot tender a defeat. The war is a holy one now. and is a prime example of why I tell everyone that there really is a "Church of PseudoScience" with many different denominations just like every other "religion".

      reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 6 Sep 2018 @ 5:59am

    https://supreme.justia.com/cases/federal/us/562/223/#F12

    ...Congress sought to provide generous compensation to those whom vaccines injured—as determined by an "expert" compensation program...

    To allow a jury in effect to second-guess those determinations is to substitute "less expert" for "more expert" judgment, thereby threatening manufacturers with liability (indeed, strict liability) in instances where any conflict between experts and nonexperts is likely to be particularly severe—instances where Congress intended the contrary...

    It is difficult to reconcile these potential conflicts and the resulting tort liabilities with a statute that seeks to diminish manufacturers’ product liability while simultaneously augmenting the role of "experts" in making compensation decisions...

    hampering the ability of the agency’s “expert regulators, in conjunction with the medical community, [to] control the availability and withdrawal of a given vaccine.”...

    The law charges HHS with responsibility for overseeing vaccine production and safety... "expert" public health organizations support its views and the matter concerns a medical and scientific question of great importance...

    the views of the "expert" agency along with those of relevant medical and scientific associations, all support the Court’s conclusions.

    Footnote 24
     Justice Breyer’s separate concurrence is even more explicitly policy driven, reflecting his own preference for the “more expert judgment” of federal agencies over the “less expert” judgment of juries.

    reply to this | link to this | view in chronology ]

    • icon
      Anonymous Anonymous Coward (profile), 6 Sep 2018 @ 6:42am

      Re:

      The problem I see with 'experts' is that 'experts' can be paid to establish whatever position the payer wants. Which 'expert' is to be believed?

      reply to this | link to this | view in chronology ]

      • identicon
        Anonymous Coward, 6 Sep 2018 @ 9:57am

        Re: Re:

        "The problem I see with 'experts' is that 'experts' can be paid to establish whatever position the payer wants. Which 'expert' is to be believed?"

        Good question... when faced with the 2nd Amendment which 'expert' do you believe?

        The Founding fathers or some lawyer that has years experience using people ignorance against them? I am assure you that no one actually cares, they only care about what is politically expedient for them and this problem is no different.

        reply to this | link to this | view in chronology ]

      • identicon
        Talmyr, 7 Sep 2018 @ 1:30am

        Re: Re:

        Which is why having established reputations through open-access publishing or reviewing might help.

        reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 6 Sep 2018 @ 6:04am

    HHS - No accountability

    https://thefreethoughtproject.com/hhs-vaccine-safety-reports-lawsuit/

    Lawsuit Proves HHS Hasn’t Filed Required Vaccine Safety Reports with Congress IN 30 YEARS.

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 6 Sep 2018 @ 8:50am

    While i am not enamoured of the pharmaceitical industry, that is some thinly veiled anti-vaxxer dreck in the comments right there.

    Are you going to compensate everyone who dies of childhood measels because they got a placebo or were waiting 20 years for a study? Vaccine i juries do happen. But people do not die in the millions because of them.

    Also, this has zero to do with peer review in journals.

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 6 Sep 2018 @ 11:03am

    Oh boy .. here come the anti-vaxxers

    reply to this | link to this | view in chronology ]


Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here
Get Techdirt’s Daily Email
Use markdown for basic formatting. HTML is no longer supported.
  Save me a cookie
Follow Techdirt
Techdirt Gear
Shop Now: Techdirt Logo Gear
Advertisement
Report this ad  |  Hide Techdirt ads
Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Chat
Advertisement
Report this ad  |  Hide Techdirt ads
Recent Stories
Advertisement
Report this ad  |  Hide Techdirt ads

Close

Email This

This feature is only available to registered users. Register or sign in to use it.