Techdirt's think tank, the Copia Institute, is working with the Trust & Safety Professional Association and its sister organization, the Trust & Safety Foundation, to produce an ongoing series of case studies about content moderation decisions. These case studies are presented in a neutral fashion, not aiming to criticize or applaud any particular decision, but to highlight the many different challenges that content moderators face and the tradeoffs they result in. Find more case studies here on Techdirt and on the TSF website.

Content Moderation Case Study: Twitter Removes Account For Pointing Users To Leaked Documents Obtained By A Hacking Collective (June 2020)

from the reporting-on-hacking dept

Summary: Late in June 2020, a leak-focused group known as "Distributed Denial of Secrets" (a.k.a., "DDoSecrets") published a large collection of law enforcement documents apparently obtained by the hacking collective Anonymous.

The DDoSecrets' data dump was timely, released as protests over the killing of a Black man by a white police officer continued around the nation neared their second consecutive month. Links to the files hosted at DDoSecrets' website spread quickly across Twitter, identified by the hashtag #BlueLeaks.

The 269-gigabyte trove of law enforcement data, emails, and other documents was taken from Netsential, which confirmed a security breach had led to the exfiltration of these files. The exfiltration was further acknowledged by the National Fusion Center Association, which told affected government agencies the stash included personally identifiable information. While this trove of data proved useful to activists and others seeking uncensored information about police activities, some expressed concern the personal info could be used to identify undercover officers or jeopardize ongoing investigations.

The first response from Twitter was to mark links to the DDoSecret files as potentially harmful to users. Users clicking on links to the data were told it might be unsafe to continue. The warning suggested the site might steal passwords, install malicious software, or harvest personal data. The final item on the list in the warning was a more accurate representation of the link destination: it said the link led to content that violated Twitter's terms of service.

Twitter's terms of service forbid users from "distributing" hacked content. This ban includes links to other sites hosting hacked content, as well as screenshots of forbidden content residing elsewhere on the web.

Shortly after the initial publication of the document trove, Twitter went further. It permanently banned DDoSecrets' Twitter account over its tweets about the hacked data. It also began removing tweets from other accounts that linked to the site.

Decisions to be made by Twitter:

  • Should the policy against the posting of hacked material be as strictly enforced when the hacked content is potentially of public interest?
  • Should Twitter have different rules for “journalists” or “journalism organizations” with regards to the distribution of information?
  • How should Twitter distinguish “hacked” information from “leaked” information?
  • Should all hacked content be treated as a violation of site terms, even if it does not contain personal info and/or trade secrets?
  • How should Twitter handle mirrors of such content?
  • How should Twitter deal with the scenario in which someone links to the materials because of their newsworthiness, without even knowing the material was hacked?
Questions and policy implications to consider:
  • Does a strict policy against "distributing" hacked content negatively affect Twitter's value as a source of breaking news?
  • Does the mirroring of hacked content significantly increase the difficulty and cost of moderation efforts?
Resolution: While DDoSecrets' site remains up and running, its Twitter account does not. The permanent suspension of the account and additional moderation efforts have limited the spread of URLs linking to the apparently illicitly-obtained documents.

Hide this

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: content moderation, ddosecrets, hacking, journalism, leaks, reporting
Companies: twitter


Reader Comments

Subscribe: RSS

View by: Time | Thread


  • identicon
    Anonymous Coward, 16 Sep 2020 @ 3:57pm

    Can't link to hacked information?

    The same can be said about the Panama Papers, and even the Pentagon Papers.

    Does Twitter bar links to those papers, or reporting about those papers? If not, why not?

    The Pentagon Papers pose a particular problem in this regard: They were just as governmental secret as the DDoSecrets trove. Prior to 2011, they were every bit as classified. Twitter had 5 years there where they theoretically should have enacted the same policy on the earlier leak.

    So sure, they don't have to play with policy about them today. But 10 years ago?

    reply to this | link to this | view in chronology ]

    • icon
      PaulT (profile), 16 Sep 2020 @ 11:27pm

      Re: Can't link to hacked information?

      "Does Twitter bar links to those papers, or reporting about those papers? If not, why not?"

      I can't speak to the actual policy, but I'd assume that at some point they get classed as historical data rather than current hacks.

      "Twitter had 5 years there where they theoretically should have enacted the same policy on the earlier leak."

      Is the policy the same, or has that changed in the intervening years?

      reply to this | link to this | view in chronology ]

      • icon
        That One Guy (profile), 17 Sep 2020 @ 10:24am

        Re: Re: Can't link to hacked information?

        I can't speak to the actual policy, but I'd assume that at some point they get classed as historical data rather than current hacks.

        Which would completely gut the usefulness of them. 'People had evidence of at-the-time ongoing corruption and violations of laws and rights, but we didn't allow links to that evidence to be posted. Now that several years have passed and it's all a moot point we will allow those links, assuming anyone cares at this point, just in case they want to see what was going on back then.'

        reply to this | link to this | view in chronology ]

  • icon
    That One Guy (profile), 16 Sep 2020 @ 4:20pm

    Shooting your trustworthiness, and users, in the back

    The first response from Twitter was to mark links to the DDoSecret files as potentially harmful to users. Users clicking on links to the data were told it might be unsafe to continue. The warning suggested the site might steal passwords, install malicious software, or harvest personal data. The final item on the list in the warning was a more accurate representation of the link destination: it said the link led to content that violated Twitter's terms of service.

    There is a huge difference between 'following this link might compromise your computer and all that entails' and 'this link leads to content that violates our TOS', and mixing those two up is a great way to screw Twitter's users over by making them less likely to take any such warnings seriously.

    If the content is a TOS violation then mark it as such or pull it if you really don't want to be associated with it, if it's malicious then don't allow the link in the first place, but conflating the two is a terrible practice that they should have known better about.

    reply to this | link to this | view in chronology ]

  • icon
    K`Tetch (profile), 17 Sep 2020 @ 2:03pm

    Want the weirdest thing?
    They also did it with the Bolton book and it's torrent links.
    I responded to pointing to the fact that the content in question is up on TPB.
    That tweet is still there

    Weird eh?

    reply to this | link to this | view in chronology ]


Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here



Subscribe to the Techdirt Daily newsletter




Comment Options:

  • Use markdown. Use plain text.
  • Make this the First Word or Last Word. No thanks. (get credits or sign in to see balance)    
  • Remember name/email/url (set a cookie)

Close

Add A Reply

Have a Techdirt Account? Sign in now. Want one? Register here



Subscribe to the Techdirt Daily newsletter




Comment Options:

  • Use markdown. Use plain text.
  • Make this the First Word or Last Word. No thanks. (get credits or sign in to see balance)    
  • Remember name/email/url (set a cookie)

Follow Techdirt
Essential Reading
Techdirt Insider Chat
Recent Stories

Close

Email This

This feature is only available to registered users. Register or sign in to use it.