Research Shows Twitter Was Missing Known Child Sex Abuse Material
from the this-is-bad dept
Soon after Elon Musk took over Twitter, he insisted that stopping child sexual abuse material (CSAM) was his top priority, and while some of his fans insisted that he had magically done so, the fact is that he fired nearly the entire team that was handling that issue, meaning that CSAM was running rampant on the site, and the company seemed to be doing little about it.
I’m guessing that all of the stories about this resulted in the folks at the Stanford Internet Observatory (SIO) researching how well Twitter was handling known CSAM images. As you may know, the “standard” for most big sites is to use a tool managed by Microsoft called PhotoDNA, which has hashes of a large database of known CSAM images, as determined by the National Center on Missing and Exploited Children (NCMEC). PhotoDNA has its issues, but the one thing it’s generally pretty good at is catching and stopping attempts to reupload images in its database.
So, SIO ran an experiment, in which they hooked up a PhotoDNA system to search Twitter and see if it found any such known images. The team at SIO wasn’t checking the images, but had them sent to NCMEC.
Making sure you have no PhotoDNA matches on your site is basically table stakes for any decently large internet platform that hosts images or video. If you can’t stop images in PhotoDNA, you’re failing, badly. And Twitter failed badly.
In just over two months, from March 12 to May 20, the researchers’ system detected more than 40 images posted to Twitter that were previously flagged as child sexual abuse material, based on a data set of roughly 100,000 tweets, said David Thiel, chief technologist of the Stanford Internet Observatory and a co-author of the report.
The appearance of the images on Twitter was striking because they had been previously flagged as child sexual abuse material, or CSAM, and were part of databases companies can use to screen content posted to their platforms, the researchers said. “This is one of the most basic things you can do to prevent CSAM online, and it did not seem to be working,” Thiel said.
Dealing with CSAM beyond PhotoDNA is a much bigger challenge, but the fact that the company couldn’t even do the basics correctly is terrifying.
In a thread on Bluesky, Renee Diresta, who worked on the research noted that they tried to reach out to Twitter to alert them that their PhotoDNA setup was missing stuff, but initially couldn’t find anyone to talk to, which is another strikes against Elon’s trust & safety team, as basically every mid- to large internet company has at least someone who knows people at SIO. It’s bad if a company doesn’t.
Eventually, the SIO team had to find a “third-party intermediary” to reintroduce them to Twitter, and somehow that finally got someone at the company to pay attention and fix the issue.
Having no remaining Trust and Safety contacts at Twitter, we approached a third-party intermediary to arrange a briefing. Twitter was informed of the problem, and the issue appears to have been resolved as of May 20.
Again, there are reasons why you have a strong trust & safety department, and that includes being able to deal with illegal content like CSAM. Yet, despite Musk claiming it was the company’s top priority, they completely fell down on the job.
Filed Under: csam, ncmec, photodna, sio, stanford
Companies: twitter
Comments on “Research Shows Twitter Was Missing Known Child Sex Abuse Material”
Is the Problem Truly Solved?
I’m doubtful that the problem’s truly been solved. The way Musk’s Twitter seems to work, they probably just made sure to get rid of all the stuff the research found, then called it a day. If they run the research again in a few months, they’ll probably find a bunch of CSAM then, too.
Of course they won’t be able to afford to run it again, with Twitter’s insane API costs, so the CSAM will just stay on Twitter going forward.
Re:
Well the good news is that you don’t need to use the API. You just grab the tweets directly from the website.
The tradeoff will be having a Musk threatening you with a lawsuit for daring not to pay him for getting that training data for your algorithm
This comment has been flagged by the community. Click here to show it.
Another day, another scurrilous piece by Double-M attacking Musk and Twitter. Elon owning Twitter is for TD just like Trump being POTUS was for the disgraced mainstream media.
Can’t wait to read the next one!
Re:
Some people — you, for example — are only alive because breathing doesn’t require any thought.
Re:
“Another day, another scurrilous piece by Double-M attacking Musk and Twitter. Elon owning Twitter is for TD just like Trump being POTUS was for the disgraced mainstream media.”
Yup- Seems Conservative News Network was so disgraced recently that their shiny new MAGA CEO is now looking for employment.
could it be that twitter owns Elon? .. just askin.
Re: Re:
wut?
Re: Re: Re:
wut what?
This comment has been flagged by the community. Click here to show it.
Re:
Mike bashing Twitter is basically the only reason I read Techdirt anymore. They’ve been suckered by the AI ghouls, blasted straight past the wingnut fringe on trademark/copyright stuff, and the stories about law enforcement’s contempt for the law are just too depressing. I guess it really does take all sorts.
Re: Re:
Care to elaborate?
This comment has been flagged by the community. Click here to show it.
Re: Re: Re:
Not really. Every time I point out that I’d quite like to be paid for my work I get accused of having brain parasites.
Re: Re: Re:2
…gaslighted nobody mentally competent, ever.
This comment has been flagged by the community. Click here to show it.
Re: Re: Re:3
Hi Toomster. Still grasping for a second insult, I see.
Re: Re: Re:4
Still struggling to recover from your messy divorce from reality we see, BDAC.
Re:
If “Twitter is unable to mitigate its CSAM problem” would’ve been news prior to Musk buying Twitter, for what reason should that story be ignored now that Musk owns Twitter?
Re: Re:
To be fair they might have considered ‘the site has a bunch of known CSAM on it they can’t be bothered to remove’ a non-story before he took over too, though given what that would say about them…
Re:
Were you dropped on your head as a child? Is that what’s wrong with you?
Re: You played yourself son
And here you are giving him the views…
Re:
Yeah, it’s like these people can’t so anything immoral or illegal without people talking about it! If only there was a way to avoid such criticism, such as by not doing illegal or immoral things?
'Who cares about that, I've got banned people to invite back!'
Always worth remembering that no discussion of the staff Elon let go/drove off when he took over is complete without mentioning it included those tasked with finding and removing CSAM, because apparently he didn’t/doesn’t consider that a high priority job.
This comment has been flagged by the community. Click here to show it.
40! Terrifying! You’re such an idiot. All this because Twitter no longer stops people from stating (correctly) that transwomen are men.
Re:
🤫 🐣 the 👫👭👬are 🗣️
Re:
There simply is no excuse for the number being more than, say, 3 if I’m being generous. Unlike most forms of moderation, this is an extremely simple thing for systems to do (it’s literally just hashing the image and comparing it to a database of hashes of images), and it should be done before the post gets uploaded. It is basically the only bit of prepublication screening that almost every social media site does simply because of how severely prosecuted mere possession of this sort of material is. No amount of context changes the fact that it is criminal.
Keep in mind, the researchers were using the least effective method of screening for child porn that is considered acceptable. In other words, there are a lot of false negatives. That means that, while 40 were caught, the system could reasonably have gone through hundreds of others that were improperly excluded. The number really should be 0.
Failing such a basic test is simply unacceptable, period. This should have been an easy perfect score, yet it was far from it.
This comment has been flagged by the community. Click here to show it.
Re: Re:
CSAM claims are nothing for a smokescreen to target men educating boys on the enactment of true love between males, which is established as superior to the behavior of breeder nuclear families whose sexual intercourse is grounded in the primitive instinct of passing on genetics. The Greeks had no need for this, which is why pederasty was a celebrated behavior and considered perfectly natural, just as nature and evolution intended. People who complain about CSAM are Catholic priests and closeted “straight” people who are actually gay that want a monopoly on that sweet sweet shotacon content.
Re: Re: Re:
None of this is true. Indeed, most gay non-Catholics also oppose CSAM, and anti-CSAM laws and efforts generally target straight, bisexual, and lesbian pedophiles just as much as gay ones. And the Ancient Greeks’ practice did not target underage boys, specifically, but simply men in general, and child marriage to girls was also allowed.
Re: Re: Re:2
“None of this is true. Indeed, most gay non-Catholics also oppose CSAM”
As do Catholics, to be fair. The issues with that church are usually down to the management and not the flock, although it can vary across different dioceses.
The problems are usually that boys are generally left alone and expected to fend for themselves whereas girls are more aggressively defended or accompanied, so pedos have more access to them. That’s been used as an attack on people who prefer adult members of the same sex for a long time, but I suspect it’s more like prison rape (you don’t have access to women, so men it is) than what they try to use to attack people in monogamous adult relationships.
Re: Re: Re:3
Nah, fuck them all anyway for letting it happen.
Making fun of idiots who believe in an imaginary friend is just what we do. What are those child molesters going to do about it? Pray that we stop?
Re:
Hang on, there’s child porn on twitter because they stopped moderating anti-trans rhetoric? Or are you saying people are researching the child porn because of the anti-trans rhetoric? Just trying to make some sliver of sense out of this, even though that seems futile.
Re: Re:
The interesting thing is that these people never acknowledge the existence of trans men. I think that reveals a few things they don’t want to admit.
Re: Re: Re:
Hywoman just can’t stand the idea that a woman might be ten times the man that he is, but it’s only a matter of time he gets replaced. And good fucking riddance.
Re:
Hi Hywoman.
Here’s your reminder that this is Pride Month, and you’re going to get fucked.
Re: Re:
Here’s your reminder that you’re less welcome here than Hyman.
Re: Re: Re:
Not likely. I’ve seen Hyman called a terrorist multiple times.
There is no low that can be stooped to in order to make sure that the likes of Hyman Rosen are intolerated.
Of course there’s an excuse. It’s computer programming. Programmers make mistakes. Things go wrong.
Re:
Programmers are more likely to make more and worse mistakes if you fire 80% of them and make everyone who is left too afraid to point out foreseeable problems.