Content Moderation Case Study: Twitter Removes Account For Pointing Users To Leaked Documents Obtained By A Hacking Collective (June 2020)
from the reporting-on-hacking dept
Summary: Late in June 2020, a leak-focused group known as “Distributed Denial of Secrets” (a.k.a., “DDoSecrets”) published a large collection of law enforcement documents apparently obtained by the hacking collective Anonymous.
The DDoSecrets’ data dump was timely, released as protests over the killing of a Black man by a white police officer continued around the nation neared their second consecutive month. Links to the files hosted at DDoSecrets’ website spread quickly across Twitter, identified by the hashtag #BlueLeaks.
The 269-gigabyte trove of law enforcement data, emails, and other documents was taken from Netsential, which confirmed a security breach had led to the exfiltration of these files. The exfiltration was further acknowledged by the National Fusion Center Association, which told affected government agencies the stash included personally identifiable information. While this trove of data proved useful to activists and others seeking uncensored information about police activities, some expressed concern the personal info could be used to identify undercover officers or jeopardize ongoing investigations.
The first response from Twitter was to mark links to the DDoSecret files as potentially harmful to users. Users clicking on links to the data were told it might be unsafe to continue. The warning suggested the site might steal passwords, install malicious software, or harvest personal data. The final item on the list in the warning was a more accurate representation of the link destination: it said the link led to content that violated Twitter’s terms of service.
Twitter’s terms of service forbid users from “distributing” hacked content. This ban includes links to other sites hosting hacked content, as well as screenshots of forbidden content residing elsewhere on the web.
Shortly after the initial publication of the document trove, Twitter went further. It permanently banned DDoSecrets’ Twitter account over its tweets about the hacked data. It also began removing tweets from other accounts that linked to the site.
Decisions to be made by Twitter:
- Should the policy against the posting of hacked material be as strictly enforced when the hacked content is potentially of public interest?
- Should Twitter have different rules for ?journalists? or ?journalism organizations? with regards to the distribution of information?
- How should Twitter distinguish ?hacked? information from ?leaked? information?
- Should all hacked content be treated as a violation of site terms, even if it does not contain personal info and/or trade secrets?
- How should Twitter handle mirrors of such content?
- How should Twitter deal with the scenario in which someone links to the materials because of their newsworthiness, without even knowing the material was hacked?
Questions and policy implications to consider:
- Does a strict policy against “distributing” hacked content negatively affect Twitter’s value as a source of breaking news?
- Does the mirroring of hacked content significantly increase the difficulty and cost of moderation efforts?
Resolution: While DDoSecrets’ site remains up and running, its Twitter account does not. The permanent suspension of the account and additional moderation efforts have limited the spread of URLs linking to the apparently illicitly-obtained documents.
Filed Under: content moderation, ddosecrets, hacking, journalism, leaks, reporting
Companies: twitter
Comments on “Content Moderation Case Study: Twitter Removes Account For Pointing Users To Leaked Documents Obtained By A Hacking Collective (June 2020)”
Can't link to hacked information?
The same can be said about the Panama Papers, and even the Pentagon Papers.
Does Twitter bar links to those papers, or reporting about those papers? If not, why not?
The Pentagon Papers pose a particular problem in this regard: They were just as governmental secret as the DDoSecrets trove. Prior to 2011, they were every bit as classified. Twitter had 5 years there where they theoretically should have enacted the same policy on the earlier leak.
So sure, they don’t have to play with policy about them today. But 10 years ago?
Re: Can't link to hacked information?
"Does Twitter bar links to those papers, or reporting about those papers? If not, why not?"
I can’t speak to the actual policy, but I’d assume that at some point they get classed as historical data rather than current hacks.
"Twitter had 5 years there where they theoretically should have enacted the same policy on the earlier leak."
Is the policy the same, or has that changed in the intervening years?
Re: Re: Can't link to hacked information?
I can’t speak to the actual policy, but I’d assume that at some point they get classed as historical data rather than current hacks.
Which would completely gut the usefulness of them. ‘People had evidence of at-the-time ongoing corruption and violations of laws and rights, but we didn’t allow links to that evidence to be posted. Now that several years have passed and it’s all a moot point we will allow those links, assuming anyone cares at this point, just in case they want to see what was going on back then.’
Shooting your trustworthiness, and users, in the back
The first response from Twitter was to mark links to the DDoSecret files as potentially harmful to users. Users clicking on links to the data were told it might be unsafe to continue. The warning suggested the site might steal passwords, install malicious software, or harvest personal data. The final item on the list in the warning was a more accurate representation of the link destination: it said the link led to content that violated Twitter’s terms of service.
There is a huge difference between ‘following this link might compromise your computer and all that entails’ and ‘this link leads to content that violates our TOS’, and mixing those two up is a great way to screw Twitter’s users over by making them less likely to take any such warnings seriously.
If the content is a TOS violation then mark it as such or pull it if you really don’t want to be associated with it, if it’s malicious then don’t allow the link in the first place, but conflating the two is a terrible practice that they should have known better about.
Want the weirdest thing?
They also did it with the Bolton book and it’s torrent links.
I responded to pointing to the fact that the content in question is up on TPB.
That tweet is still there
Weird eh?