Cryptographer Tells European ISPs How EU’s Client-Side Scanning Proposal Will Make Everyone Less Safe

(Mis)Uses of Technology

from the obvious-problems,-oblivious-legislators dept

It’s not as though we really need any more evidence that client-side scanning is a bad idea. Apple decided to be a pioneer and immediately discovered the world wasn’t exactly waiting for it to become a market leader in privacy invasion.

We don’t need more information. We know breaking encryption results in broken encryption — something that’s useless when it comes to ensuring privacy and security. We know client-side scanning will result in an entirely new, entirely unpleasant can of digital worms being opened.

We know this. Unfortunately, those making the rules (for the most part) either pretend these concerns aren’t legitimate or have decided the nominal gains in law enforcement efficiency will outweigh the collateral damage done to millions of innocent people.

Fortunately, plenty of experts in the field are speaking up about the threat posed to users’ privacy and security by legislative proposals that mandate weakened encryption and/or client-side scanning in hopes of combating CSAM.

Respected cryptographer Matthew Green was given the opportunity to speak to the European Internet Service Providers Association (EuroISPA). His presentation — published in full on his site — spells out in plain English the many problems and side effects of mandated client-side scanning, including the fact that it won’t just affect hundreds of millions of European residents, but everyone all over the world utilizing services affected by the proposed legislation. (All emphasis in the original.)

Some have argued that the new proposal is not about encryption at all. At some level these people are correct. The new legislation is fundamentally about privacy and confidentiality, and where law enforcement interests should balance against those things. I have opinions about this, but I’m not an EU citizen. Unfortunately this is a fraught debate that Europeans will have to have among themselves. I don’t envy you.

What concerns me is that the Commission does not appear to have a strong grasp on the technical implications of their proposal, and they do not seem to have considered how it will harm the security of our global communications systems. And this does affect me, because the security of our communications infrastructure is not localized to any one continent: if the 447 million citizens of the EU vote to weaken these technical systems, it could affect all consumers of computer security technology worldwide.

Fortunately, his remarks were delivered to service providers, who may at least have some grasp of the technical realities of the EU Commission’s proposal. This is essential, because the Commission pushing this legislation clearly doesn’t comprehend these realities. Nor does it appear interested in being fully informed of these issues.

I have read the Impact Assessment authored by the Commission, and I hope I am not being rude to this audience when I say that I found it deeply naive and alarming. My impression is that the authors do not understand, at a purely technical level, that they are asking technology providers to deploy systems that none of them know how to build safely. Nor has the Commission consulted people with the technical and scientific expertise that would be needed to make this proposal viable.

For those of you who, I assume, are not members of the EU Commission and wish to learn more about the technical ramifications of client-side scanning, as well as the problems it introduces when it comes to matching hashes to detect illicit content, I encourage you to click through and read Green’s presentation. He does an excellent job breaking down technical issues into something even casual readers can understand. He also goes further, explaining how these weaknesses would be exploited by malicious people and sexual abusers of children to cause further harm to other service/platform users, not to mention the actual victims of sexual abuse.

For example, introducing flaws in encryption to enable client-side scanning creates these problems, which currently don’t exist under end-to-end encryption:

This ability to selectively disable encryption creates new opportunities for attacks. If an attacker can identify the conditions that will cause the model to reduce the confidentiality of your encryption, she can generate new — and apparently harmless — content that will cause this to happen. This will very quickly overwhelm the scanning system, rendering it useless. But it will also seriously reduce the privacy of many users.

A mirror version of this attacker exists as well: he will use knowledge of the model to evade these systems, producing new imagery and content that appear unchanged, but that these systems cannot detect at all.

That’s only part of the foreseen issues client-side scanning will introduce into the internet ecosystem. There’s a whole range of other issues that can’t possibly be foreseen because the systems used to detect illicit material will continue to evolve, along with the laws that will likely expand to cover content that isn’t CSAM and, consequently, will have no starter set of known, verified hashes to match uploaded content with.

The EU Commission doesn’t appear to have a problem with any of this. It apparently would prefer to go live and fix it in post. Here’s how Green describes the Commission’s approach:

I hope that the Commission will rethink its hurried schedule and give this proposal enough time to be evaluated by scientists and researchers here in Europe and around the world. We should seek to understand these technical details as a precondition for mandating new technologies, rather than attempting to “build the airplane while we are flying in it”, which is very much what this proposal will encourage.

That’s what’s happening here: people who don’t know how to build planes nor the desire to understand how they work are insisting on putting this claptrap contraption in the air as soon as legislatively possible. And while the wreckage scatters across the world, the pilots of this doomed vessel with be sure to celebrate any CSAM-related arrests as worth the privacy/security sacrifices forcibly extracted from millions of Europeans and, ultimately, billions of people all over the world.

Filed Under: client side scanning, csam, encryption, privacy, surveillance