If you enjoy badly written bipartisan protectionist tech legislation designed primarily to coddle giant U.S. companies under the xenophobia-tinged breathless pretense of privacy and national security, you are really going to enjoy the next twelve to twenty-four months.
While the U.S. drowns in corrupt kakistocracy, the Chinese are making significant market inroads in everything from AI to EVs. That’s resulted in U.S. companies applying greater and greater pressure on U.S. lawmakers to simply ban Chinese goods. The Trump administration’s adoption of this policy has been a hot and sloppy protectionist mess, incompetently implemented and unsubtly racist.
Automakers are particularly worried about cheaper, better Chinese EVs making their way to the U.S. So under the banner of the misleadingly named Alliance for Automotive Innovation, they’re pressuring U.S. lawmakers to enact a ban on Chinese EVs. You know, because they’re very worried about privacy and national security:
“Right now, Chinese automakers are dumping subsidized vehicles with connected software and hardware around the world,” John Bozzella, CEO of the group, said in the letter seen by CNBC. “This hasn’t happened inside the U.S. yet, but given the scale and urgency of this threat, we urge you to enact a Chinese vehicle, software and hardware ban before adjourning this year and make this policy the law of the land.”
“Enacting a permanent ban on Chinese vehicles and high-risk hardware and software in the 119th Congress will send a clear and bipartisan message that China’s strategy to dominate global automotive manufacturing will be met with a national security policy response from the American government,” Bozzella said.
So for one, I like how the auto industry throws the word “subsidized” around as if they haven’t enjoyed generations’ worth of their own pointless subsidies. Two, the U.S. auto industry has some of the worst privacy standards and ratings of any industry in America, and sell the entirety of your driving, personal, and behavior data to any old random asshole in a country too corrupt to pass privacy laws.
Failing to secure your own vehicles and fighting tooth and nail against any privacy safeguards… then ranting incoherently about the threat of Chinese tech on U.S. shores is not serious policy. Our failure to regulate data brokers or pass modern privacy laws means the Chinese simply buy this same data from any of dozens of dodgy companies already, making a lot of this stuff lazy pantomime.
We’ve seen this before: Democratic lawmakers in Michigan recently tried to ban Chinese EVs from even visiting the state, suggesting that automakers are afraid of Americans even getting to look at overseas alternatives. The justification (by folks who are are, again, completely absent when it comes to any sort of domestic U.S. privacy standards) is they were just very concerned about U.S. consumer privacy.
I maintain that ideally you allow Chinese companies to compete in the U.S. market, but you fund, staff, and legally protect your labor, consumer, competition, and environmental regulators so that companies are all genuinely competing on a level playing field. You boot or penalize obvious bad actors on privacy, security, competition, and consumer protection, both foreign and domestic.
U.S. corporate giants don’t much want that, given it means more oversight, more competition, and diminished quarterly returns. So what we often get instead is a sort of a corrupt-fueled incompetent rank protectionism that’s highly performative but still broadly harmful.
And while you could theoretically implement protectionism in a way that’s coherent, the U.S. is too corrupt to do that. So what you get is stuff like the TikTok ban, which was driven by years of hysteria about Chinese spying and propaganda, only to result in a bipartisan array of lawmakers shoveling TikTok off to Trump’s billionaire autocrat friends, which was not any net improvement.
Or you get stuff like the “race to 5G,” which involved U.S. policymakers being told that the only way to keep pace with Chinese 5G was to give U.S. telecoms less oversight, more pointless subsidies, and approval for their terrible mergers (the U.S. lost the “race to 5G” in terms of reach, network quality, and affordability then immediately just… stopped talking about it).
As a backdrop we have a U.S. corporate press that’s incapable of expressing how badly any of this is going in practice (often because affluent media ownership supports the administration and its mindless deregulation), resulting in this strange disconnect between material reality and the performance lawmakers put on to convince themselves they’re doing serious and useful policy.
If U.S. policymakers cared about privacy and national security they’d pass a meaningful modern privacy law (with powerful penalties for U.S. companies or executives), and they’d regulate data brokers. If they cared about national security, they’d eject Donald Trump from the body politic. Unless they’re doing these things, they’re not really worth taking seriously on privacy or national security.
With cheaper Chinese AI models threatening U.S. tech giants’ dreams of software automation walled garden dominance, you can expect all of this sort of performative dysfunction to get much much dumber, supported by the press and the kind of folks who’ll talk your ear off over cocktails about how much they love free markets and the kind of innovation forged in the furnace of real competition.
In a handful of known cases, large social media companies have privately pushed back against Immigration and Customs Enforcement (ICE) subpoenas when the agency tried to unmask anonymous users who tracked immigration activities or criticized the government.
As ICE engages in a pattern of illegal and chilling investigations, any resistance is welcome. But social media companies can do more. When companies receive these unlawful subpoenas, they should be clear with the public that they will not hand over the data unless a court compels them to do so. In addition, companies themselves can take the government to court to challenge these unlawful subpoenas on behalf of their users.
Publicly challenging these unlawful subpoenas in court has the dual purpose of protecting individual users who may lack the resources or know-how to challenge a subpoena on their own, while also discouraging ICE from issuing similarly unlawful subpoenas in the future.
Companies have a responsibility to protect the privacy of their users. That responsibility does not end simply because companies wish to avoid the ire of this administration—which has sought to chill other powerful institutions like news outlets, law firms, universities, and non-profits.
ICE Has Issued Many Unlawful Subpoenas
ICE has sent hundreds of subpoenas to large technology companies like Google, Meta, and Reddit seeking basic subscriber information like name, email address, IP address, and session times.
Some of these subpoenas have targeted people who engaged in protected activity—like tracking immigration actions, criticizing the government, or attending a protest. People have a First Amendment right to document law enforcement activities and criticize the government online, without retaliatory government investigations. This right has become more important as immigration agents have engaged in invasive, unconstitutional, and sometimes violent conduct.
In a handful of cases, users themselves have successfully pushed back. After receiving notice of these subpoenas, users have challenged them in court, relying on pro-bono lawyers from groups like the ACLU or Civil Liberties Defense Center. Companies have been largely absent from these court proceedings.
Private Pushback from Meta and Reddit
While not appearing in court, companies like Meta and Reddit have sometimes pushed back behind the scenes.
For example, on September 11, 2025, ICE sent administrative subpoenas to Meta seeking to unmask users who ran Instagram and Facebook accounts that tracked immigration activity in Pennsylvania. On September 19, 2025, Meta’s Law Enforcement Response Team told ICE that the agency did not have the “statutory authorization” to seek the records. It asked for more detail about the investigation and said “Meta will take no further action with respect to this summons until it receives this information.” Later, Meta informed ICE that it planned to notify the users about the subpoenas, since no gag order had been obtained. The government disclosed this information in one of EFF’s Freedom of Information Act lawsuits against ICE and other agencies.
On October 3, 2025, Meta notified the user about the subpoena. Despite its private pushback, Meta told the users it would comply with the subpoenas unless they mounted a court challenge within 10 days—which they did with the help of the ACLU. Ultimately, ICE withdrew the subpoenas when it became likely that ICE would lose the case in court.
In another example, Reddit documented its pushback in a transparency report released a few months ago. Reddit reported that in the second half of 2025, the company received three Department of Homeland Security (DHS) subpoenas seeking account information from 11 users who posted content critical of ICE. In the report, the company stated that “Reddit objected to these legal demands because the users appeared to be engaged in protected activity under the First Amendment, and law enforcement withdrew their requests.” The company reported that most other DHS subpoenas it received appeared to be routine.
A Tech Company Model for Public Resistance
EFF’s demand that technology companies do more to protect their users is not unprecedented. Twitter (now X) did so successfully in the first Trump administration.
On April 6 2017, Twitter went to court to challenge a DHS subpoena that sought to unmask a Twitter account named “@ALT_USCIS,” which frequently criticized the administration’s immigration policies. Twitter challenged the subpoena on both statutory and First Amendment grounds. A day later, DHS withdrew the subpoena and Twitter dismissed the case. The incident led to an inspector general investigation, which criticized a tactic that DHS is still engaged in.
In other circumstances, companies have also gone to court to protect their users and shield themselves from burdensome legal process. In 2013, Microsoft challenged a search warrant for the content of emails stored on servers outside the United States. In 2015, Apple challenged a court order to break the security of its iPhone during an investigation into the San Bernardino shootings. And in 2007, Yahoo challenged the constitutionality of government requests at the Foreign Intelligence Surveillance Court.
Even the records HIPAA does cover can be shared, sold or handed to the government in ways that might surprise you.
This gap in protection matters more than ever because the U.S. government is pushing hard to gather health data domesticallyand abroad. This is happening even as a growing body of research shows that the safeguard which these efforts to collect data lean on – anonymizing data by removing identifying information to make it difficult to trace back to an individual – is far weaker than officials claim.
As a professor of law at Indiana University, I study health information privacy and medical data regulation, which includes tracing how sensitive health information moves among clinics, government agencies and law enforcement. As a co-investigator on a federally funded study about opioid prescribing, I rely on health data in my own research. I appreciate its value for science, and I also see the danger of collecting it without meaningful safeguards.
Limits of medical privacy
HIPAA gives you several rights: You can see your health records, demand corrections and expect that a covered provider will not casually disclose your information.
The statute is also thick with additional exceptions. In practice, much of your health information can be shared through these many open doors. And once data is sent outside the system covered by HIPAA, the HIPAA limits fall away.
For instance, prescription drug monitoring programs, which every state now operates, assemble detailed logs of who filled which controlled substance prescription and when. Federal law enforcement can often access these logs with a self-issued administrative subpoena – an order that doesn’t require a judge’s approval or oversight.
These programs have expanded beyond opioids into a dragnet that shares health data across state lines, exposing patients who seek reproductive or gender-affirming healthcare to surveillance far from home.
Health records can flow to many destinations under different rules. A given disclosure might feel more like a violation depending on who decides where it can go and who can then see it.
RFK Jr.’s push to access Americans’ health records
Since the spring of 2025, Health and Human Services Secretary Robert F. Kennedy, Jr. has sought federal access to Americans’ medical records to investigate whether vaccines cause autism. The scientific community has studied this question for decades and has shown decisively that they do not.
According to KFF Health News, HHS has been courting state health information exchanges – the little-known systems that let hospitals and clinics swap detailed, identifiable patient records – and asking how those records might be used for vaccine research. One proposal floated by state organizations would give HHS data on 90% of Americans’ medical records by 2028. In Nebraska, millions of federal grant dollars have flowed to a statewide health information exchange nonprofit that has cooperated with the effort.
The concern is not that the government should never collect health data. It is that meaningful safeguards have not kept pace with the scale of collection and capabilities of modern data analytics.
In seeking access to Americans’ medical records for a vaccine and autism study, HHS has declined to say how many states are involved, what data it collects, who can see it or how it will be protected.
Building a comprehensive repository to chase a question that science has already answered inverts the logic of research. Usually a hypothesis justifies the data collected, rather than the reverse.
Collecting identifiable records for tens of millions of people in a single database also creates a target for breaches, secondary uses that no one consented to and abuses by current or future administrations with different priorities.
Decades of computer science research undercuts that promise. A study published in Nature in June 2026 sharpened the point, showing that in this age of artificial intelligence, stripping identifiers from patient records to protect identity does not protect all patients equally.
The researchers audited AI diagnostic models trained on clinical data, including chest X-rays, electrocardiograms and electronic health records. They asked whether an outsider could tell if a particular person’s data had been used to build the model. For instance, confirming that someone’s record helped train a cancer-prediction tool can reveal that that person has cancer. This exploit is known as a membership inference attack.
The research team found that while the average risk of being identified from data stripped of identifying information often looked reassuringly low, some patients faced near-certain reidentification The burden fell unevenly: Underrepresented groups, sorted by race, insurance status or diagnosis, were most at risk. Those most exposed were frequently already most vulnerable to discrimination.
Researchers have long established that removing identifiers from rich datasets does not reliably protect the people in them, and that identification gets easier the more information you have. Today’s AI technology makes it possible to carry out these attacks remotely and quickly.
The same privacy problems, exported
The U.S. government’s appetite for health data does not stop at the border. As ProPublica reported in June 2026, the State Department has been conditioning lifesaving aid to African nations on access to their citizens’ health data.
Under the Trump administration’s global health plan, Uganda agreed to give the United States real-time access to nine of its health data systems for seven years, including the central repository of the nation’s health information and the system managing individual electronic medical records, in exchange for up to US$1.7 billion over five years, a sum that shrinks each year and falls below prior U.S. support. Kenya struck a similar deal; Zambia, Zimbabwe and Ghana walked away from the initial terms.
The U.S. government has promised that the data will be aggregated and anonymized, but privacy experts warn that the agreements are vague and omit standard limits on how much data is taken and how it can be used. A Ugandan digital rights lawyer called the choice his country faced the essence of digital colonialism: Accept the deal and risk exploitation, or refuse it and watch people die.
The common thread
Domestic records collection and foreign data-for-aid deals rest on the same faith that anonymization neutralizes the risk of pooling sensitive health data.
The evidence says otherwise. This does not mean health data should never be gathered or studied, but I believe that the reassurances deserve skepticism, the safeguards deserve scrutiny, and the people whose bodies generated the data deserve a say. To safeguard privacy, a government seeking sensitive medical records should have to show why it needs them and how the safeguards it relies on hold up.
Privacy law was built for a world where data resided in filing cabinets. Governments from Kalamazoo to Kampala now operate in a world where even an anonymized digital record can point back to you.
From the very beginning of this recent obsession with identifying everyone online (yes, they like to call it “age” verification, but it always ends up as identity verification), we’ve been pointing out that it was a huge privacy nightmare waiting to happen. Or maybe it wasn’t waiting. Maybe it was already happening.
This week a massive new data breach has been revealed that should put the nail in the coffin for the idea that any sort of age or identity verification could be safe. 153 million scans of drivers licenses easily available based on this breach, with more being added all the time. Literally on the day it was revealed (and right before the site was taken down) it added another 400,000 records to its available database.
There is no safe age verification. There is no age verification that doesn’t put people at risk.
Last year, Eric Goldman wrote the definitive piece on how all of these technologies — no matter what they tell you — are huge privacy risks, but people are still living in denial. This is despite the numerous examples we’ve had in just the past few years of verification providers and their customers having massive data breaches.
The latest comes to us via Brian Krebs, who reports on a massive breach of scanned IDs — more than 153 million drivers licenses from people across the US and Canada, now for sale on the dark web:
A new identity theft service launched on the dark web this week is selling digital scans of more than 153 million drivers licenses from people in the United States and Canada. Based on interviews with individuals whose licenses are available for purchase on this service, it appears to be siphoning images collected by a widely-used identity verification company based in Louisiana. KrebsOnSecurity also has learned that the New Orleans field office of theFederal Bureau of Investigation(FBI) today launched an official inquiry into the source of the images.
Krebs traces the breach back to an ID verifier that appears to be used by many companies, including Hertz, the rental car company. It appears to not be limited to them either, as he checked with a number of people who were in the database, and by looking at the date they were added alongside their calendars, found examples of other people who shared their ID at places like a pot dispensary.
That company turns out to be IDScan.net, based in Louisiana, which has contracts with thousands of dispensaries, not to mention Hertz, FedEx, and Target. And while Krebs is focused on how many of the leaked IDs are connected to real world businesses, it’s worth noting that IDScan.net is also doing age verification for a bunch of tech companies, has a page tracking state age verification laws and company implementations, and even has written positively about laws like KOSA, the Kids Online Safety Act, that would effectively require age verification.
So, yes, we have a company that is a big player in the age verification space, talking up age and identity verification laws, that appears to have had a long-standing ongoing leak of every ID it scanned.
Yiiiiiikes.
And, of course, like all age and identity verification providers, IDScan has spent years talking up how secure it keeps all this data, even as every single record appeared to be leaking in realtime. Here’s their “Trust Center” page which is still up days after the hack was revealed:
That’s the company that spent over a year leaking 150 million drivers licenses in real time, explaining “how we protect data, maintain system reliability, and earn the confidence of our customers and their users.” Might be time to update that page.
But also, this should be a massive warning to everyone pushing for age verification laws. You can have a “trusted” company in the space who brags about all the certifications it has. It’s in “compliance” with the GDPR, the CCPA, and every other law. It is “transparent” about its “privacy practices” and how its “sensitive identity data is handled responsibly” and…. for over a year it’s been leaking all of those sensitive records.
And it appears no one internally at the company noticed.
As Krebs makes clear, the breach included many, many millions of records and ID scans that were being swiped in real time by the hackers who breached the system:
The people behind Nexus claim the license images are coming from an active breach at “a major identity verification company” whose customers include multiple Fortune 500 companies.
The record totals listed by the Nexus identity theft service. The number of drivers license records increased by nearly 400,000 in the span of just 24 hours.
“We have been continuously exfiltrating new data for over a year into our private database,” the service enthused in its introductory post on Exploit. “Records are available to preview before purchase with pertinent information redacted. Customer photos are displayed if available.”
Indeed, over the past 24 hours, the number of drivers license records listed as available in Nexus has increased by nearly 400,000, suggesting that freshly stolen license data is being harvested and uploaded to this serviceon a semi-regular basis.
And the exposed records aren’t just random members of the public. Krebs found the driver’s license of the sitting Secretary of Defense sitting in there for sale:
A bargain! Only $100 to get a scan of the Secretary of Defense’s driver’s license.
Anyway, each time we highlight a breach people play it down and insist that mandating age verification is perfectly safe and nothing to worry about. Yet here’s one of the largest identity verification companies in the country, with a pipeline so wide open that hackers had a real-time feed of every government ID it scanned, for over a year, without anyone at the company noticing.
Krebs spoke to a security researcher at Cybera, named Larry Baldwin, who talks about how this kind of data can do real damage:
Baldwin said the Nexus identity theft service presents multiple serious security and privacy threats, noting that state-issued drivers licenses are commonly used as proof of one’s identity when opening new lines of credit. Baldwin said the service could also dangerously expose many people who do not wish to be found but who cannot meaningfully change their appearance (or at least not enough to fool today’s AI-based image matching tools).
This category of people, he said, includes those fleeing domestic violence, and even people who have been assigned a whole new life and identity as part of the federal government’s witness protection program, which is generally reserved for criminal defendants in racketeering and conspiracy investigations who agree to cooperate with federal authorities.
“Just when it seems like we’re making some headway in improving authentication controls through drivers license verification systems, this happens and the very thing those improvements are dependent on are compromised,” Baldwin said.
At this point, anyone still supporting age verification requirements, especially claiming it’s for “child safety,” should have to answer for all the millions of people put needlessly at risk due to data breaches like this.
You cannot do age or identity verification safely. It always creates some sort of record and that set of records will always become a target. That’s what happened here. And it’s what will happen with any such systems.
Meta’s settlement with 52 state attorneys general is a bad deal for all internet users, and especially for teens. That’s what we said the day the settlement was announced.
In this post, we go through the Settlement’s provisions in detail and explain why that is so, including how:
The Settlement embeds age assurance technology and age-gates into Meta’s social media products and requires all users—minors and adults—to undergo a rights-threatening age estimation process
The Settlement places severe restrictions on Teens that can only be modified by the Teens’ parents and only then in exchange for giving their parents a ton of information about their online community and usage;
The Settlement seems to empower the attorneys general to enforce Meta’s content restriction on “age inappropriate content,” categories that Meta itself has had trouble administering without excluding information about sexuality, sexual and reproductive healthcare, and abortion medication;
The Settlement actually requires Meta to collect, analyze, and retain more information about its Teen users—when the pressure should have been on Meta to diminish its surveillance capitalism.
Note: A chunk of the settlement addresses unauthorized users under age 13, and Meta’s obligations to comply with the Children’s Online Privacy Protection Act. Meta policy has banned users under 13 since the company opened to the public in 2006. Aside from the age assurance frameworks that support both those and the other parts of the Settlement, the under-13 provisions are not addressed in this post. Those provisions essentially require Meta to detect and delete all under-13 accounts.
Further note: All U.S. states are parties to the Settlement except Florida, New Mexico, and Texas. The Settlement includes D.C., American Samoa, Guam, Northern Mariana Islands, and Puerto Rico.
Age Gates Reinforced By Age Estimation Technology
In the Settlement, Meta agrees to age-gate Instagram and Facebook, thus making age gates a legal mandate. And Further, Meta will now enforce these age gates with “age assurance” technology, ditching its previous practice where the person signing up for the services self-attests to their birthdate. This concession firmly embeds deeply flawed age estimation technology into the online experience of millions of people around the world. First and foremost, the age verification setup seriously threatens online anonymity and privacy for everyone, as we’ve said before. The Technology also just adds a layer of creepiness into the user of any service. In the Settlement, Meta pledges to, within one year, apply one or more age assurance methods to each Instagram or Facebook user in the states and territories that joined the Settlement. [P. 10, §II.A.1]
1. Age Assurance Framework. Within one (1) year of the Effective Date, Meta will adopt an age assurance framework (“Age Assurance Framework”), wherein it will apply one or more age assurance methods developed by a third party and licensed to customers (“Commercially Available Age Assurance Methods”) orage assurance methods developed by Meta (“Proprietary Age Assurance Methods”) (collectively, “Age Assurance Methods”) to each Meta SMP user in the Settling States. For the purposes of this Section II.A, an age assurance method developed or acquired by Meta that uses the same or functionally identical technology and methodology to a Commercially Available Age Assurance Method shall be treated as a Commercially Available Age Assurance Method. The Age Assurance Framework must include Age Assurance Methods to evaluate whether a Meta SMP user is a Teen User or U13, as described in Section II.A.6. New users of Meta SMPs who have not yet had their age assessed by an Age Assurance Method pursuant to Meta’s Age Assurance Framework shall receive the Default Protections pursuant to Section II.A.10 of this Agreement.
Those methods might include both commercially available products, as well as proprietary age estimation process Meta might have or develop. Meta also pledges to consider age signals from Google and Apple operating systems and app stores. [§II.A.5] Meta has previously advocated for age assurance requirements to fall on Google and Apple rather than on individual services.
This age assessment essentially dumps users into one of three age-range buckets: 18+, 13-17, and under-13. Users under 13 have long been barred from Meta products, but this Settlement creates new obligations to search for and detect users who may have said they were older.
For those estimated to be over-18, the Settlement guarantees no direct benefit to you: no privacy protections, no greater user controls for your own accounts, no dent in Meta’s surveillance capitalism.
Those estimated to be 13-17 years old will be limited to Teen User accounts.
Those estimated to be under-13 will lose their accounts altogether.
Those who open new accounts will have two weeks to submit to age estimation, and if they decline to do so, Meta is now required to treat them as a Teen User by default, even if they self-identify as being 18 and older. [P. 18, §II.A.10.b]
(b) Fourteen (14) days or more after creating a Meta SMP account, Meta SMP users who have not yet had their age assessed by an Age Assurance Method pursuant to the Age Assurance Framework shall be treated as Teen Users for the purposes of this Agreement regardless of their stated age, except that Meta SMP users with a stated age of 18 years old or older shall receive the protections described in Section II.A.10.a.ii.
What about people with existing accounts, who are well past that two-week period to submit to age estimation? Will they also be defaulted to Teen User status if they decline age estimation? It seems so—the AGs would likely not have accepted a settlement that did not require Meta to take action against existing teen users who choose to forgo the age assurance process. Perhaps Meta will use its existing store of information about its current users as a type of permitted proprietary age assurance process? Thus, perhaps, an adult user whose Facebook account is itself older than 18 will be assessed as being over-18? Or a user who is identified as the spouse of a user who has been age-assured? But Meta can only rely on a proprietary process if it meets the accuracy standards set out in the Settlement Agreement.
How accurate does the age assurance process need to be?
The Settlement sets maximum false positive for both commercially available and proprietary age assurance methods [Pages 12-13, §II.A.6]. Within two years, each shall be no more 10% for ages 16-17 and 3% for ages 13-15.
6. Age Assurance Standards. (a) U18 False Positive Rate Thresholds. (i) Any Commercially Available Age Assurance Methods deployed by Meta for any new and existing users in the Settling States shall meet or fall below the following U18 False Positive Rates excluding method circumvention within one year of the Effective Date: 10% for minors aged 16-17 and 3% for minors aged 13-15. (ii) Any Proprietary Age Assurance Methods deployed by Meta for any new and existing users in the Settling States shall meet or fall below the following U18 False Positive Rates excluding method circumvention within 1 and 2 years of the Effective Date: (A) Year 1: Within one year of the Effective Date: 14% for minors aged 16-17 and 7% for minors aged 13-15. (B) Year 2: Within two years of the Effective Date, 10% for minors aged 16-17 and 5% for minors aged 13-15.
Notably, there is no limit indicated in the error rate for false negatives, when the process wrongly identifies an adult as being under 18. The Settlement only requires Meta to maintain an appeal process for users whose age range is wrongly assessed. [P. 17, §II.A.9]
9. Appeals Process. Users claiming to have been mis-identified as minors must be offered a Clear and Conspicuous means to appeal the decision. Decisions on all user appeals must be made in a timely manner and communicated to the user along with a basis for the decision.
The Settlement generally shows little concern for those falsely placed in its Teen User category.
Meta must also employ measures to discourage age estimation circumvention, including placing limits on the number of attempts any user might make. [P. 16, §II.A.7] As part of this, Meta agrees to proactively monitor adult accounts to determine whether a user needs to undergo additional age estimation. [P. 16, §II.A.7.c] This is just one of the ways the Settlement embeds Meta’s active surveillance of its users for the next ten years (see below for more).
(c) Incorporating a proactive monitoring system that requires users to undergo an additional Age Assurance Method where a user is determined, including based on their conduct on Meta SMPs, to have likely circumvented the Age Assurance Method and is: (A) likely a Teen User after having been previously assessed as 18 or older; or (B) likely U13 after having been previously assessed as 13 or older. Users Meta determines are likely Teen Users after having been previously assessed as at least eighteen may choose not to undergo additional age assurance but then will be treated as Teen Users; and
Any age assurance process Meta uses must be tested annually.
Data minimization
The Settlement does have data minimization requirements for the data collected during the age assurance process. [§II.A.8] But there are numerous holes. The Settlement requires that all information obtained and retained as part of the age assurance processes thereafter be “immediately enqueued for deletion, after which it shall be deleted after a reasonable period of time.” But the Settlement defines a category of “Retainable Data” that may be retained for 90 days. This includes “metadata about the age assurance method used by the user information … where required to ensure the ongoing integrity of age assurance systems, including but not limited to the ability to identify circumvention and related patterns over time, and only for as long as it is required for those purposes.” The Settlement requires at that all data collected by Meta or its vendor be stored according to industry-standard data security measures,” a standard that unfortunately does not eliminate the risk of a data breach.
8. Data minimization and security. (a) Except as set forth herein, all data collected by Meta from users of Meta SMPs in the Settling States for the sole purpose of conducting age assurance, all data maintained from known U13s, and all data collected by a vendor for use in a Commercially Available Age Assurance Method shall be held for the minimum period required to determine a user’s age status and thereafter immediately enqueued for deletion, after which it shall be deleted in a reasonable period of time. Meta may retain (1) U13 data only to the extent required for purposes of developing, training, testing, and measuring the performance of the U13 Age Model (“U13 Data”), provided that any U13 data that constitutes Personally Identifiable Information as defined by 16 C.F.R. Part 312.2 will be protected using Meta’s highest data privacy and security standards, and (2) metadata about the age assurance method used by the user information (“Retainable Data”) only where required to ensure the ongoing integrity of age assurance systems, including but not limited to the ability to identify circumvention and related patterns over time, and only for as long as it is required for those purposes. For the avoidance of doubt, U13 Data cannot be used for purposes such as ads targeting and delivery, marketing, or algorithmic optimization efforts. Any U13 Data and Retainable Data shall be maintained at the coarsest viable granularity and cannot be used for any other purpose unless legally required. Any Retainable Data that is no longer required for the purposes set forth herein shall be deleted within 90 days. The terms above shall not pertain to the user’s stated date of birth, stated age, nor the outcome of the Age Assurance Method (e.g., “teen or adult” classification). (b) Data collected by Meta or a vendor, or transmitted by a vendor, must be collected and stored using industry-standard data security measures and as required by law, including encryption in transit and at rest. (c) The Parties agree to discuss in good faith potential modifications to this provision as necessary to permit Meta to improve the efficacy of its Age Assurance Framework while preserving the principles of data minimization and security set forth in this Section.
Restrictions For Teens (And Anyone Who Opts Out of Age-Gating)
Teen User Accounts are subject to time, feature, and content restrictions. These will be applied to these estimated to be 13-17 as well as any new user who declines to submit to the age assurance process and to existing users who decline to submit to age estimation and for whom Meta cannot ascertain that they are older than 17.
Time restrictions
Teen Users will be subject to the following time restrictions (§II.B). These measures seem to assume that most of teen’s social media use is frivolous and unserious (not that those are in and of themselves bad; the right to play is among young people’s human rights), ignoring the fact that teens use social media for school and personal research, conducting activism campaigns, and other endeavors that might naturally not fit within these time limits:
Night Access Mode – no access (except messaging) to Instagram and Facebook from Midnight to 6 AM, and no push notifications from 10 PM – 7 AM.
School mode – no push notifications from 8 AM – 3 PM Monday-Friday from Aug 15-June 15.
Daily cumulative time limit of 2 hours per day across Instagram and Facebook, resetting at midnight, excluding video and audio content at least 22 minutes long, absent artificial prolongation, defined by the Settlement as “longform content.”
“Productive pauses and notices” designed to “reduce or prevent excessive, mindless, or unintended teen usage.” This means that a teen’s usage will be monetarily paused after 60 and 90 minutes of daily cumulative use with notices sent every 15 minutes of continuous use. According to the Settlement, these productive pauses and notices will look like this:
To be clear, the ability to set time limits, blackout times, and scheduled pauses are all useful features that should be available and easy to implement for users of all ages. Such tools would have allowed teens, and all users, the ability to design their own safe experience, customized to their own needs, online. Such users controls would have recognized that teens have human rights, agency, and autonomy.
But that’s not what these restrictions are. They are not tools that give the teen users control. Rather, they are imposed, top-down, on teens and anyone else who declines to submit to Meta’s age assurance process.
Feature restrictions (§II.C-D)
Within four months of the effective date of the Settlement, Meta must offer teens an option for a non-personalized feed, which is defined as a feed of chronologically ordered posts from friends and follows. Teens will also be able to disable autoplay as part of an “optional protective settings” package. Each of these settings must be “viewable within three user gestures and clearly labeled, easy to notice, viewable without scrolling, and discoverable in an intuitive location within” the service.
Again, these would be useful user controls that should be offered to users of all ages.
By default, teens will not see the number of likes or other reactions to their posts.
Teens will also not have access to what the Settlement calls “Cosmetic Procedure Filters,” that is, “any digital filter or augmented reality effect that distorts, sculpts, redefines, or idealizes a user’s face in a way that cannot be achieved without cosmetic surgery or extreme make-up techniques.
X. “Cosmetic Procedure Filter” shall mean any digital filter or augmented reality (AR) effect that distorts, sculpts, redefines, or idealizes a user’s face in a way that cannot be achieved without cosmetic surgery or extreme makeup techniques. For the avoidance of doubt, “Cosmetic Procedure Filter” does not include: (1) Fantasy/Character Effects: Filters that change a user’s facial structure for the purpose of turning the user into a non-human or fictional character (e.g., an elf or cartoon) or an animal (e.g., a dragon or puppy); (2) Makeup/Smoothing Effects: Filters that smooth skin or alter appearance in ways that can be achieved by ordinary makeup techniques without altering the appearance of underlying physical facial structure or meaningfully changing skin tone; or (3) Parody and Exaggeration Effects: Effects that entertain users by distorting their appearance through parody, satire, or exaggerated forms (e.g., extreme visual distortions outside the scope of normal cosmetic procedures). To help operationalize this definition, the Settling States will provide Meta illustrative examples and guidance of AR effects that are Cosmetic Procedure Filters and AR effects that are not Cosmetic Procedure Filters in a letter to be sent within two (2) months of the Effective Date.
Meta has already had rules about cosmetic effects directed at teens since 2019. But the Settlement will give the states a major role in helping Meta identify what features are and are not Cosmetic Procedure Filters.
Content restrictions (P.1, §II.E, as defined by §I.C, E, F)
For content, Meta is basically pledging to continue its existing practices limiting Teen Users to age-appropriate content and accounts, to default Teen Users to age-appropriate experiences. This includes limiting access to accounts that “regularly share content that is inappropriate for teens” such as content from the following Meta community standards categories: Adult Nudity & Sexual Activity, Restricted Goods & Services, Suicide, Self-Harm or Eating Disorders.
C. “Age Appropriate Experiences” refers to content that is appropriate for Teen Users based on input from experts, parents, and teens. For the purposes of this Agreement, Age Appropriate Experiences shall mean content captured in Meta’s applicable Ages 13+ content setting, which is tied to policies inspired by movie ratings for ages 13+ and parent feedback. D. “Age Assurance Methods” shall have the meaning set forth in Section II. E. “Age Inappropriate Accounts” refers to accounts that regularly share content that is inappropriate for teens or that have account information that otherwise suggests the account is inappropriate for Teen Users. For the purposes of this Agreement, Age Inappropriate Accounts shall mean accounts that: (1) regularly share Age Inappropriate Content in the following categories: Adult Nudity & Sexual Activity, Restricted Goods & Services, Suicide, Self-Harm or Eating Disorders, or (2) have account names or profile photos or bios that suggest the account is otherwise inappropriate for minors, based on Meta’s policies for the following categories: Adult Nudity & Sexual Activity, Restricted Goods & Services, Suicide, Self-Harm or Eating Disorders. F. “Age Inappropriate Content” refers to content that is generally perceived by U.S. parents, youth experts, and teens as not being appropriate for Teen Users. For the purposes of this Agreement, Age Inappropriate Content shall mean content prohibited by Meta’s Community Standards concerning bullying and harassment; nudity and sexual activity; child sexual exploitation, abuse, and nudity; sexually explicit language; suicide, self-harm and eating disorders; graphic violence and incitements to violence; gambling; and restricted substances or goods (including illegal drug use), as well as policies specifically focused on protections for Teen Users, including those regarding high-risk viral challenges and risky stunts.
And even more worrisome, even though this is just Meta continuing its existing practices, the Settlement empowers the states to enforce its provisions. [P. 40, §IV.C.1.i; §VII.C] That means that over the next ten years, the duration of the Settlement, Meta will face the threat that a state attorney general will pursue legal action against it because it disagrees with how Meta interprets these categories of community standards, and pressure Meta to eliminate Teen User access to posts about sexuality and reproductive and sexual health. And Meta will now lack the hard-earnedFirst Amendment defenses to make its own curatorial decisions.
C. Notwithstanding anything in Sections VIII.A-B above, a Settling State may take any action, including but not limited to legal action to enforce compliance with the Agreement, without delay if the Settling State believes that a threat to the health or safety of the public requires immediate action.
The Parental Supervision Tradeoff
All of these Teen User restrictions can be modified – but only if the Teen User enrolls in the Parental Supervision that links their account to a parent’s or guardian’s account. Once their accounts are linked, Parents can modify the Teen User settings to make them less restrictive (they need the teen’s permission if they want to make them more restrictive).
And Parental Supervision comes with a huge privacy tradeoff. In exchange for designating someone as their Parent, the Parent gets a lot of information about the Teen’s use: the usernames of all of the teen’s connections, reports on how much time the Teen User spends on a Meta service, the time spent watching longform content, usernames of all those messaging with Teen User, and any evidence Meta has about suspected secondary accounts. The Supervising Parent also gets notices of the teen’s repeated searches related to suicide, self-harm and eating disorders. [P. 28, §II.G]
Parental Supervision 1. Meta agrees to provide Supervising Parents with information concerning the amount of time their Teen User is spending on Meta SMPs, including separately for time on the Meta SMP, time using the Meta SMP’s messaging features, time viewing Longform Content (to the extent excluded from the calculation of daily limits pursuant to Section II.B.3.a.i), and the usernames of the Teen User’s social connections and individuals messaging the Teen User, and usernames of any user reported by a Teen User. In addition, Instagram will notify Supervising Parents should their Teen User engage in repeated searches for terms related to suicide, self-harm, or eating disorders, and Facebook will notify Supervising Parents should their Teen User engage in repeated searches for terms related to suicide, self-harm, or eating disorders. 2. For Teen Users, Supervising Parents will be notified on a daily basis when the Teen User communicates directly with each adult user for the first time with the Teen User and shall provide a link to the adult user’s profile to provide information such as their stated hometown or city or mutual connections, to the extent the information is publicly available. 3. Upon a Teen User enrolling Parental Supervision, Meta SMPs shall prompt the Supervising Parent to review their Teen User’s settings to consider making updates to support how their Teen User spends their time on Meta SMPs. Meta shall not preselect, recommend, or encourage settings to Supervising Parents that are less restrictive than the default settings in this Agreement. 4. Meta SMPs will periodically suggest checkups for Supervising Parents to do with their Teen Users to evaluate their settings and usage. 5. Meta SMPs agree to continue to take steps designed to encourage enrollment in their Parental Supervision tools. 6. If a Teen User is enrolled in Parental Supervision, Meta shall notify the Supervising Parent if the Teen User creates or explicitly links a new secondary Meta SMP account within their Accounts Center or has been linked to a Soft Matched Account through Meta SMPs, including a link to the secondary account’s profile. Meta SMPs shall automatically apply the Supervising Parent’s approved time management settings in Section II.B and content restrictions to all explicitly linked Meta SMP accounts and all Supervised Accounts for a Teen User belonging to the same Meta SMP.
Unfortunately, Meta doesn’t have a great way to learn whether there exists a genuine parent-Teen User relationship. But it will try to get it right. [§II.G.8]
This may be workable for young people with healthy and safe relationships with their parent or guardian. But obviously not good at all for a Teen User lacking such a safe relationship.
More Surveillance, Not Less
Moreover, rather than pushing Meta away from the collection, analysis, and retention of user data, the Settlement requires Meta to do more of it for the next ten years. Several provisions of the Settlement require Meta to carefully track its users use of Instagram and Facebook for the purposes of determining whether its age assurance framework and Teen User restrictions are working as intended.
For example:
Meta promises to review detected U13 users’ friend networks to identify other possible U13 users. [§II.A.6.(b).(i).(D)]
Meta will incorporate a “proactive monitoring system” to identify possible Teen Users among those who were initially determined to be older than 17 by its age assurance process. [§II.7.(c)]
Meta pledges to utilize and improve its existing “soft matching models” that track signals such as device IDs, phone numbers, and email addresses” to identify duplicate accounts. [§II.B.6.(a)]
Meta will continue to monitor Teen Users’ activity to “regularly evaluate the prevalence of Teen User exposure to Age Inappropriate Content, and Teen User exposure to experiences that are not Age Appropriate Experiences,” [§II.E.3] and also to “regularly evaluate the prevalence of Teen User exposure to Harmful Experiences on Meta SMPs.” [§II.F.3]
Monitor Teen Users’ messaging to identify message threads with potentially suspicious accounts. [§II.F.4]
The Settlement also requires Meta to actively monitor use of its services so that it can supply data to the Independent Auditor that will now monitor its compliance with the Settlement. “The Independent Auditor is entitled to access the non-privileged information, personnel, systems, and records that are reasonably relevant and sufficient to evaluate Meta’s implementation of the Injunctive Relief Terms, including, but not limited to, access to raw data; aggregated data; information; internal documents and communications” plus information from its age assurance processes, data regarding Teen User responses to the Productive Pauses, data regarding its models for soft matching of secondary accounts, data regarding the prevalence of Teen User exposure to Age Inappropriate Content and Harmful Experiences, and to experiences that are not Age Appropriate Experiences. [§III.E]
Moreover, one of the chief threats of Meta’ surveillance is the honeypot of data it creates that may be accessible to governments for law enforcement and other investigations. Nowhere in the Settlement do the 52 attorneys general pledge to not try to access all of the data the Settlement requires Meta to collect and retain.
Meta Has To Pay The States — Establishing Norms Beyond Meta
The Settlement also includes annual payments from Meta to the states, apparently proportionate to the size of each state’s teen user base. Over the ten year-life of the Settlement, these annual payments will total over $11 billion plus the prospect of an additional $5 billion—if Meta competitors adopt the same measures.
This quirk of the Settlement incentivizes the States to pursue similar age assurance processes and at-least-as-restrictive teen user measures for Meta’s chief existing competitors for teen use, YouTube, TikTok, and Snap, and for any new service that may gain widespread teen use over the life of the Settlement Agreement. If the states are able to get Meta’s competitors to adopt the same measures, then the states will get an additional $5 billion in annual payments. That’s quite the incentive for the states to pursue litigation and regulatory measures against those companies. All of this will further entrench age assurance and age-gating as the norm across online services.
1. In the event the Contingent Monetary Payment Trigger has occurred in a Settling State, Meta shall be obligated to pay to such Settling State ten equal installments in the amount as set out in Exhibit B (each, a “Contingency Installment Payment”).
2. Following the date the Contingent Monetary Payment Trigger has occurred, the Contingency Installment Payments shall be made to the Settling State on January 15 in each subsequent calendar year of the Agreement Term as follows:
(a) At the next scheduled payment date, the Settling State shall be paid the Contingency Installment Payment for that payment date and all prior payment dates. (b) For each of the remaining payment dates, the Settling State will be paid the Contingency Installment Payment for that payment date.
3. If a Settling State fails to achieve the Contingent Monetary Payment Trigger during the Agreement Term, the Contingency Installment Payments shall be permanently forfeited by such Settling State and retained by Meta. For the avoidance of doubt, no Settling State will have an obligation to repay the Contingency Payment
The Settlement is thus a bad deal for all users of Facebook and Instagram. It normalizes age gating and age assurance for millions of internet users. It denies teens the tools to create their own safe experiences online and places their social media experience firmly under the control of either Meta or their parents. And rather than addressing Meta’s collection, analysis, and retention of data about teens’ use of Instagram and Facebook, it binds Meta to continued surveillance.
Imagine a scenario where a documentary filmmaker, in the course of making the documentary, captures some damning footage of corporate malfeasance, which she wishes to share with an investigative reporting organization anonymously. Should we be concerned that mandates on AI watermarking might reveal who she is, even if she’s not using AI at all?
Last week I pointed out some of the concerns I had with Anthropic’s AI-generated text watermarking implementation. As I explained, plenty of people use these tools for perfectly legitimate reasons. I talked specifically about non-native English speakers and some disabled communities, and how a label as binary as “some AI was used on this” inevitably lumps those uses in with all the genuinely bad ones.
A friend pointed me to a separate concern that I had not considered, from the human rights group WITNESS. I should say that WITNESS is generally supportive of AI transparency rules, and was apparently involved in the process to create the EU’s Code of Practice related to the rules that forced Anthropic to add these watermarks. But, for obvious reasons, it’s concerned about the privacy implications of these tools. Indeed, it released a fascinating report about how watermarking done badly represents a surveillance risk.
The scenario I described to open this piece comes straight from that report:
Her production software is C2PA-enabled.
She uses it because her international distribution partners require it. When she installed it, the setup asked for her name, email, and country. Standard fields. She completed them and started working.
What the setup process did not explain is that the software’s default configuration attaches her account details to the Content Credentials of every file she exports, via the CAWG identity extension. The option to disable this exists, in an advanced settings panel she has never opened, described in language that assumes familiarity with the C2PA specifications.
For most of the year this does not matter. Then, in the final weeks of production, she films something unplanned: a confrontation between managers and workers organizing without official recognition. She decides to submit the clip anonymously to a press freedom organization abroad. She exports it without checking the Content Credentials panel, because she does not know there is anything there that needs checking.
Her name travels with the file.
The report focuses on C2PA, which is the emerging standard most companies are using for non-text watermarking (for images, videos, etc.). It was put together by a bunch of the tech companies to solve their own problems regarding identifying AI-generated content. But with the EU’s AI Act and similar laws showing up, it’s getting pulled from “here’s a nifty tech solution” into “this is part of the law.” And, as the report notes, the current implementation can be abused for surveillance:
The populations most exposed are journalists, human rights defenders, and documentary filmmakers. For these groups, content provenance infrastructure creates a distinct and underappreciated surveillance surface: one that links identity to specific digital content with cryptographic precision, accumulates into detailed behavioral profiles over time, and is made harder to contest by the regulatory legitimacy surrounding it. Viewers of credentialed content face their own exposure: the act of verifying content can generate a behavioral record without their knowledge or consent.
This doesn’t mean that watermarking shouldn’t be used, but rather, as WITNESS notes, we should be aware of the risks, and seek to counter them.
The report lists multiple ways that “provenance” tools like watermarking can expose personal information. The most obvious: once watermarking is mandatory, piggybacking identity requirements on top of it becomes trivial — which, in practice, means close to inevitable:
The first is legislative and regulatory misuse. A government that understands the C2PA’s privacy surface can exploit it deliberately — through mandated identity assertions, required credentials as a condition of distribution, or convergence with national identity systems. The more likely near-term risk, however, may be a well-intentioned regulator who mandates C2PA-compliant credentials without understanding what that mandate activates. The outcome can be functionally identical to deliberate misuse.
While the report doesn’t say this quite so directly, you can see how mandates for this technology, combined with growing mandates for age or identity verification, could do real damage:
Identity can be required as a condition of creating or distributing content. A law or platform policy may require attaching personal information to Content Credentials before content can be published or distributed. The C2PA specification does not prohibit this as mandatory identity assertions may, in specific use cases, be a legitimate use of the standard. A government mandate requiring journalists to register their identity with a national authority before their content can carry verified credentials would require no modification to the specifications whatsoever, and would not be distinguishable, at the infrastructure layer, from those legitimate uses
We already have governments increasingly requiring everyone to prove their identity in some form before they can look at content. The provenance mandates are something of a mirror image: a mandate to prove who is creating the content before you can publish it. And that mandate is being dressed up as an anti-disinformation tool wrapped in a human rights cloak, making it way more difficult to push back on than a state porn-ID law. And that’s before we mention how the “AI” component leads many people who would otherwise be careful about tech mandates to scream “fuck AI, do this!”
The report also points out that content creators may not realize what information gets included in a watermark.
Personally identifiable information can be added by the user — inadvertently, or without being informed of the privacy implications of doing so. Content Credentials can carry personal information added by the creator—a name, a caption, a device identifier—without the tool surfacing what that disclosure means or who can access it. The harm is not always intentional on the part of the platform: tool design that prioritizes functionality over privacy literacy can produce the same outcome as deliberate data collection. A photographer including personal attribution to an image may not realize that information will travel permanently with the file, accessible to anyone who inspects the manifest.
Even in cases where people think they’re being careful, a pattern may still emerge that reveals sensitive information:
Identity can emerge from patterns across a body of published work.
Identity may become recoverable not from an individual manifest but from correlating assertions across a body of work over time— locations, timestamps, device identifiers, behavioral signatures—none of which individually crosses a sensitivity threshold, but which together build a detailed profile. For example, a state actor scraping a manifest store to map the movement patterns of an activist photographer across months of published work would not need access to any single sensitive file.
And perhaps worst of all, the final risk they highlight is that simply the act of verifying the provenance of some form of media requires interacting with third parties that may reveal some amount of information:
Engaging with Content Credentials exposes creator and audience behavior to third parties. Engaging with Content Credentials — whether as a creator signing content or as an audience member verifying it — can expose behavior to third parties. On the creation side, signing operations that require external connections for timestamping, certificate status checks, or manifest store submission generate server-side records linking the creator’s device, location, and timestamp to a specific piece of content, without any disclosure that this is occurring. On the verification side, depending on implementation, remote validation may require the viewer’s device to contact an external server directly, generating a logged request that records who verified what, from where, and when. In neither case does the affected party have awareness that this is happening or any means of refusing it: unlike cookies or tracking pixels, the C2PA specifications include no consent mechanism, no opt-out, and no disclosure requirements. A journalist signing footage before publication may unknowingly leave a server-side trace of that act. A reader who encounters a suspicious image on social media and verifies its provenance may unknowingly send a request associating their IP address, approximate location, and timestamp with that specific piece of content. At scale, across a platform or a jurisdiction, these logs become a map of who is creating what and who is reading what, where and when.
While the descriptions of the surveillance threats from the tech are good, what drives it home are some of the fictional scenarios that are absolutely worth reading. There’s a story of a government passing an “anti-disinformation” law, which then enables that government to track down a reporter exposing government malfeasance, because her identity is tied to her digital tools via its digital provenance requirements. In another scenario, a local reporting outfit working on an investigative piece partners with a foreign media org to hide its own involvement — only to have it revealed by the watermarking tech.
Or the story of an anonymous online video producer, who doesn’t realize that despite efforts to protect his identity, these provenance mandates actually reveal to everyone who he is. Perhaps the most terrifying is the human rights worker documenting war crimes, taking massive privacy and security precautions, but is ratted out by the tech in ways that are difficult to predict:
The state actor does not need a surveillance program to make the connection. They need two things that are already publicly available. The first is the organization’s own archive. In regions where field staff safety is less of a concern, the organization signs its content with its organizational identity. It is standard practice, and a source of institutional credibility with the tribunals and monitoring bodies it works with. That archive is public, verifiable, and searchable. It establishes, unambiguously, that this organization uses this specific tool. The association between the tool signature and the organization’s name is not inferred. It is proven, repeatedly, by the organization’s own publishing practice in contexts where they had no reason to hide it.
The second is the content credential metadata ecosystem. Services that index C2PA manifests, aggregating records from published content across platforms, make the tool signature searchable across a body of work. The conflict zone footage, submitted to the monitoring body and entering a semi-public record, carries the same tool signature as dozens of other pieces of content the organization has published under its name elsewhere.
The tool signature in the conflict zone footage matches the tool signature in the organization’s public archive. The organization’s known field presence does the rest. The credential record the organization designed to protect its staff contains, in the tool signature alone, a thread that leads directly back to them, and they placed that thread in the public record themselves, in good faith, in a different context entirely. The anonymity set was the user base of that tool, in that region, in that period, and that number was small enough to matter.
One of the problems of anonymity software today is that if not enough people are using it for everything else, your mere use of it alone may reveal things about you. That’s what the last paragraph of this scenario highlights.
That scenario also calls out another vector of concern: as more and more media comes with C2PA credentials (or other watermarks) attached, we’re going to get more and more aggregation by third parties, which opens up yet another vector of surveillance. After so many years of concerns about the aggregation of private information — especially in the EU with the GDPR — you’d hope that regulators would be more careful not to create another way to amass huge collections of data on each of us.
Instead, the EU spent all these years building an entire (somewhat annoying!) “consent” regime centered on the idea that a third party shouldn’t be logging what you looked at on the internet without first getting your permission. So it’s a bit odd for this very same regulatory apparatus to then push an infrastructure that might hand a lot of private information over to aggregators… just in a more secretive manner.
Again, none of this is to say that watermarks are inherently bad. There are many cases where they are incredibly useful. WITNESS’s own report leads off by saying that it is “increasingly necessary” and a “part of restoring trust in the information environment.” It also has many suggestions for how to build better, privacy preserving tools to do this better.
But a transparency tool that doubles as a tracking layer for journalists, human rights defenders, and the people reading their work is not much of a win for the information environment it’s supposed to be restoring.
This is a point we keep hammering on about tech policy, and especially about the sorts of technology mandates that have become so popular these days. It is really, really hard to look at an entire ecosystem and see how the pieces interact — but that’s the job when you’re writing rules that everyone has to build to. Mandates that might increase competition can decrease privacy and security. Mandates that might increase transparency can decrease competition or security. Almost every decision has tradeoffs.
We still need to make those decisions, but we should do so with our eyes open regarding the tradeoffs, and figure out the best ways to minimize the harms while increasing the benefits. Unfortunately, as it stands, it’s not clear that regulators have really understood all the potential downsides regarding mandated watermarking transparency yet.
When I wrote about the concern of watermark mandates last week, a lot of people were quick to dismiss them. “AI sucks and no one should use it” was the attitude of many commenters. But it’s not just about AI, as hopefully the examples in this article highlight. The filmmaker using her regular tools or the human rights worker documenting war crimes shouldn’t lose their anonymity because these mandates were designed to stop people from making a fake video of a politician.
There’s a hell of a lot of work left to do to get this right. Currently, the EU’s AI Act mandates a label designed to help you check whether the content you’re consuming was generated with the help of AI tools. But depending on how it’s implemented, that setup can create real problems. The very act of checking the provenance of an image or video can put your own IP address, your location, and a timestamp in some third party’s server log, tied to that media. Worried regulators mandated that the provenance tracking exist. Now we’re all going to have to deal with the fallout.
Even as the wider EU was grinding through a long, drawn-out process to figure out which regulatory levers to pull on kids’ safety online, France decided to YOLO it earlier this summer by jumping at the chance to ban all social media for kids under 15. This kind of thing has become popular with out of touch adults in the grips of an ongoing moral panic, since Australia led the way after gambling companies pushed the ban as an alternative to banning gambling ads. Australia’s ban isn’t doing very well, with the majority of kids figuring out how to work around it, and those being left out being the most marginalized and in need of community.
And yet, countries around the globe have all suddenly decided — some based on reading Jonathan Haidt’s badly reasoned book — that they must do this too.
France was the first in the EU, with President Emmanuel Macron gleefully treating France’s willingness to rush in with little thought or understanding as a selling point:
“France is leading the way in Europe in protecting our children and our teenagers,” Macron said. “We will keep on going.”
He wanted the ban to be implemented in mere months, just as kids returned to school.
But that’s all run into a bit of a stumbling block known as the French Constitutional Council, which has said the ban is an unconstitutional attack on kids’ rights to free expression and communication. The Council also flagged a second problem: you can’t enforce an under-15 ban without making every user, adults included, prove their age. The law demanded exactly that, without defining a single condition, limit, or technical standard for how that verification would work.
A court in France on Friday shot downa bill seeking to ban access to social media for under-15s from September — a major blow to President Emmanuel Macron that raises fundamental questions about efforts to protect kids on the internet.
The Constitutional Council, which reviews the constitutionality of French legislation, said the restrictions in the bill disproportionately infringe on minors’ right to freedom of expression and communication.
Reading through the Google translation of the actual ruling, it’s pretty short and to the point. Similar to how the First Amendment requires any restriction on speech to be narrowly tailored to a specific government interest, here the Council says the ban is way too broad and not based on any specific, narrowly defined harm, though it admits that the aims of protecting children are certainly legitimate:
Furthermore, while the established prohibition does not apply to online encyclopedias, educational or scientific directories, or platforms for developing and sharing free software or open-source educational digital projects, the exceptions provided for in the contested provisions remain limited. In particular, these exemptions do not cover collaborative services for sharing leisure, information, or mutual aid content, online communication applications, or online games with strong collaborative and social features, nor do they cover online social networks which, while not inherently educational, are created in connection with educational activities.
Thus, the prohibition established is likely to apply to online communication services whose risks to the health and safety of minors, relating in particular to their content or mode of operation, are not established.
In short, the law goes way too far in issuing a blanket ban of all children, no matter what the circumstances. If you have a legitimate, well-defined problem, come up with a narrowly tailored solution. The French government rushed this one through with little concern for things like that.
It also leaves little room for parents to decide for their own kids what’s appropriate:
… neither the contested provisions nor any other provision sets out the conditions under which the holders of parental authority or the legal representative of the minor, duly informed of the potential risks and safeguards presented by the services concerned, may, in the child’s best interests and in the exercise of their duties under the law, decide to lift the prohibition, limit its scope or authorise access to certain services.
Thus, the prohibition established does not give rise to any particular assessment of the risk to the minor, taking into account in particular his age, his degree of maturity, his family situation as well as the nature of the service concerned.
Also, the age verification attack on privacy is important to recognize:
By prohibiting access for any minor under fifteen years of age to certain online services, the contested provisions imply, in themselves, that any person, even an adult, must prove their age before accessing them.
The Council further notes that the law makes no real effort to figure out how one might implement age verification in a manner that protects the privacy rights of adults.
Of course, having hitched his own legacy to this thing, there’s no way Macron lets it go quietly:
In a statement late Friday, the French presidential office said the government would not be giving up on the bill. It has set a new target date for spring 2027, which coincides with when Macron will leave office.
The statement said Macron “has instructed the Prime Minister to work, as quickly as possible, on a legally sound draft that takes into account” the court’s decision. The ruling hinted at what would make the age restriction align with fundamental rights: giving parents more flexibility.
Politico also spoke to Peter Craddock, a Brussels-based attorney who works on social media regulation, who notes that any other EU country attempting a similar blanket ban is likely to run into exactly the same wall:
“The reasoning is actually equally relevant internationally, throughout the EU, because this fundamental freedom is not specific to France,” he said.
Which is a useful reminder that the freedom of expression problem here isn’t a quirk of French constitutional law. It’s baked into the whole approach — and no amount of “but it’s for the children” framing makes it go away. That’s even more true of the problems with age verification requiring the scanning of everyone’s ID, which is an even touchier subject in large parts of the EU than elsewhere.
Really, though, the bigger, more important message here should be to slow down. What’s incredible is that for all of the political and media class whining that social media is some rogue experiment on our children, none of them seem to consider that abruptly trying to block all social media from kids is just as much an experiment, and one that might have equally damaging effects.
Why not wait and see how the Australian ban actually works in practice? The early results are a mess. I get that Macron and other politicians want headlines and a legacy to point at, but it would be nice if they actually followed what the research shows and looked at how the early experiments of these bans have worked out.
So what France produced here was a total rush job that sacrificed the expression rights of every teenager in the country, the ability of parents to make their own judgment calls regarding their own kids’ access to information and — as a cherry on top — the privacy of every adult who would now need to prove their age at the door to the internet. Thankfully, the Constitutional Council caught all three.
The easiest way to tell that Flock Safety doesn’t really care that its massive network of ALPR (automatic license plate reader) cameras is being abused by cops to stalk their exes and harass residents is to judge it by its actions. When confronted with court case after court case detailing these allegations (and those are just the cops who actually got caught!), the company says two things: first, we’re not personally responsible for the actions of “bad apples” and second, some vague “fixes” are on the way at some indeterminate point in the future.
The other way you can tell Flock doesn’t care about anything but future profitability is this: when cities pull the plug on Flock systems due to local backlash, Flock has — one more than one occasion! — decided to to simply turn the cameras back on.
Dayton, Ohio ended its contract with Flock following public complaints and some apparently forbidden sharing of data with federal immigration officers. But the city had to resort to covering Flock cameras with garbage bags to prevent further recordings since even the PD seemed unsure of whether or not it could actually control this function and Flock itself remained deliberately vague about when (or if!) it would be showing up to remove the recording equipment that contractually still belonged to it.
Last week, for example, the mayor of Menominee, Wisconsin said that Flock cameras in the city “have been activated without city council approval.”
[…]
[Evanston, Illinois] previously ordered Flock to shut down 19 cameras (18 stationary and one flex camera that can be attached to a squad car) provided by the company and put its contract with Flock on a 30-day termination notice on Aug. 26. The company took down 15 of the 18 stationary cameras by Sept. 8, only to reinstall all of them by Tuesday.
Neat, huh? That’s on top of an audit performed by the South Carolina Secretary of Transportation that found more than 200 unpermitted Flock cameras operating on public roads.
Last week, police discovered Flock had restored power to the cameras without notifying the town.
The discovery, detailed in a Littleton Police Department notice this week, led the department to order the cameras remotely powered down, disable system login access for officers, and confirm a Flockwork order was in place to remove the cameras from utility poles “as soon as possible.”
That’s pretty fucked up. That’s Flock saying that not only is the customer never right but the customer shouldn’t be allowed to make their own decisions. It’s Flock unilaterally deciding that the rest of the nation’s users of Flock’s ALPR systems shouldn’t be deprived of access to Littleton’s cameras just because the entity paying for its services decided it no longer wanted to do business with Flock.
The Littleton PD did everything it could to abide by the vote and the town’s wishes. Flock, however, went the other way seemingly simply because it could. While PD officials have said this might have been the result of some miscommunication between the PD and Flock, it would seem an existing Flock work order to remove cameras would not have honestly resulted in this:
According to the statement, Littleton Police Chief Douglas Landry met with a Flock technician following the incident, and that “given the volume of vandalism and tampering incidents the company has dealt with nationally, it would not necessarily be treated as unusual on [Flock’s] end” to have a camera need to be turned back on unexpectedly.
Do what now? Flock was told to remove the cameras. Instead, Flock decided this meant it could flip the on/off switch as often as it wanted to until it actually got around to removing the cameras, due to vague concerns about backlash Flock has absolutely earned.
And I can guarantee you whatever things Flock is saying at this point, it’s only saying because it got caught. I have yet to see an incident where Flock has informed cities or PDs of inadvertent activation or illegal access. In every case, it’s always Flock making excuses after the fact for things it could have done more of to prevent (illegal access) or abusive things it did itself (re-activating/re-installing cameras) that contradicted the intent of city residents and their governments. These are not the actions of a trustworthy tech provider. These are the actions of a company that clearly believes it cannot be fucked with because it has managed to corner this market.
Recent reports have raised alarm about the use of PatronScan, an ID-checking and face-scanning system, at multiple LGBTQ+ bars in San Francisco’s Castro neighborhood. Much of the attention has focused on reports that the system photographs patrons as they enter venues and questions about whether those images are used for facial recognition.
A broader privacy concern also deserves scrutiny. For years, PatronScan has marketed itself not just as an ID-verification tool, but as a system that allows bars and clubs to identify patrons, keep records about them, and share information across venues. As one news article published in 2019 documented, PatronScan built a network that allowed participating bars to flag patrons and share information about them with other establishments.
And in California, it’s not at all clear how PatronScan’s business model of scanning IDs and sharing the information from those scans with other bars comports with the law. California’s ID privacy law, which was amended in 2018 to add ID “scans,” states that no businesses shall “retain or use” any information from a scanned ID card except for limited purposes such as to verify age, comply with a legal requirement, or prevent fraud.
Californians should be deeply concerned about businesses that collect information from government-issued IDs and use it to build databases about where people go, whom they associate with, and whether they should be allowed into other public gathering places. That concern is especially strong in LGBTQ+ spaces, which have long served as refuges for people to go without being tracked, monitored, or put on lists.
We reached out to Patronscan with questions regarding their practices and their views on California ID law. They referred us to their published FAQ question “Is Patronscan privacy compliant in California?” which claims that the use of Patronscan kiosks is legal in California. They also said “Patronscan does not do facial recognition in North America, or any kind of automated analysis of the ID or the live photo image.”
The California Legislature Has Investigated PatronScan’s Business Model
In 2018, the California Legislature published bill analyses (on that year’s AB 2769) that went into detail about PatronScan’s business. Reviewing PatronScan’s own materials, the California Senate Judiciary Committee found that the company had collected and retained information on 561,087 customers in Sacramento alone during the first five months of 2018—a remarkable figure for a city whose population had only recently topped 500,000.
Lawmakers also found that at that time, PatronScan retained information for at least 90 days or longer in some cases, shared information among participating bars, and maintained bans that lasted an average of more than 19 years. A PatronScan “Public Safety Report” used 10,000 scans collected on a single day to report on “where customers live, how far they have traveled, and how many different venues the customers patronized.”
This was not simply checking IDs at the door. PatronScan was building a database.
An immigrants’ rights group, the Coalition for Human Immigrant Rights (CHIRLA), wrote about its concern at the time with these growing ID databases, saying that “placing individuals on a database that labels them a “threat to public safety” has “significant immigration consequences that could lead to deportation, revoking of current status, or denial of future immigration relief.”
Today, PatronScan states that it retains personal information about all customers for 21 days, and about flagged customers for up to five years. This includes the customer’s name, date of birth, photograph, gender, and zip code. It also includes the dates and times that the customer entered particular bars. Such databases are a grave privacy threat. Personal data is routinely stolen by thieves, misused by a company’s employees, seized by government agencies, and diverted to new purposes by a company’s executives.
California Law Still Bans ID-Scan Databases, And Bars Should Follow That Law
In 2018, California lawmakers closed what they viewed as a loophole. Existing law already prohibited businesses from retaining or using information obtained when they “swiped” a driver’s license, except for the narrow purposes of legal requirements (like a judicial warrant) or “preventing fraud, abuse, or material misrepresentation.”
After reviewing companies like PatronScan, the Legislature amended the law to make clear that the same restrictions that apply to businesses that “swipe” ID cards also apply when those IDs are “scanned.” PatronScan opposed that change, arguing it wanted to preserve the ability to share information among bars so participating venues could decide whether to admit patrons.
The bill became law anyway. Yet PatronScan continues to market and sell a system that apparently retains information from scanned IDs, and allows participating venues to flag patrons and share information across its network.
At a minimum, that raises serious questions about how those practices fit with California’s existing ID privacy law. Bar and nightlife venue owners who utilize PatronScan should think twice about its effects on their customers, and consider going back to standard, visual ID checks. These physical checks have been effective at keeping underage patrons out of 21-and-over venues for decades, and don’t present the serious privacy dangers of creating a private database of bar patrons.
For venues serving vulnerable communities like immigrants or the LGBTQ+ community, the stakes of using this technology are even higher. It’s disappointing and alarming to see some of California’s more well-known LGBTQ+ nightlife spots instead lining up as PatronScan’s early adopters. A venue cannot claim to be a safe space while feeding its patrons’ data to a third party database. These businesses should reject PatronScan, return to the standard ID checks that every other bar has been able to utilize, and prove to their customers that their privacy and security still matters.
The Senate Commerce Committee is set to consider S. 737, the SCREEN Act, a sweeping age-verification bill that would require online services to verify users’ ages before they can access any sexually explicit content. If this bill passes, it will force millions of adult internet users to give up their anonymity, privacy, and security before they access lawful speech.
Unlike many state-age verification laws—which have been harmful in their own right—the SCREEN Act has no requirement that a significant portion of the website consist of sexually explicit content that is harmful to minors. The bill requires nearly any service hosting even a single piece of sexually explicit content to verify the ages of its users. The result is that the bill would apply not only to adult content sites like PornHub or OnlyFans, but also streaming services like Netflix, and social media platforms like Reddit, Discord, or Bluesky, if they host any adult content.
The SCREEN Act does not merely require users to attest they are adults. It specifically states that “requiring a user to confirm that the user is not a minor shall not be sufficient.” In practice, that means platforms would have to verify users’ ages using methods tied to their real identities. Providing proof of age online is dramatically different, and far more invasive, than showing your ID at the door to a bartender or bouncer. In the physical world, the bouncer at the door looks at your ID card, confirms you’re old enough, and gives it back to you. Under the SCREEN Act, the “bouncer” will be a digital age-verification service that captures your personal information and saves it to a database for an unspecified amount of time.
The consequences of the bill won’t be limited to minors. If websites and apps are expected to reliably identify teenagers, adults will be asked to prove they are adults.
Even worse, the SCREEN Act is a privacy and data security nightmare. One provision of the bill requires services to take reasonable steps to protect the data collected and to not maintain for longer than is necessary. But these are terribly weak protections that impose no meaningful collection, use, or retention limits on services collecting people’s private information.
In other words, the third parties tasked with verifying a user’s age on a platform could sweep up a lot of personal info they don’t actually need and then could use that information for any number of purposes, so long as they deem their actions reasonable. Companies would then be allowed to keep the information users have been compelled to turn over for as long as possible, raising security and privacy issues along the way.
The SCREEN Act Attacks Your Right To Use VPNs
The SCREEN Act also targets virtual private network (VPN) users and providers. The bill requires covered websites to verify users’ ages based on their IP addresses unless the service can determine that the user is outside the United States, and specifically requires age verification on traffic coming from known VPN addresses. In practice, this discourages the use of VPNs and proxy servers, which millions of people rely on for legitimate purposes such as protecting personal privacy, securing public Wi-Fi connections, safeguarding journalists and activists, and preventing data tracking.
VPNs mask your real location by routing your internet traffic through a server somewhere else. When you visit a website through a VPN, that website only sees the VPN server’s IP address, not your actual location. It’s like sending a letter through a P.O. box so the recipient doesn’t know where you really live. VPNs are a privacy and security tool used by millions of internet users every day, and their use should not be treated as suspect. It is particularly galling that the SCREEN Act forces users who intentionally take steps to protect their privacy to identify themselves.
The SCREEN Act creates onerous age-verification rules that will block adults from accessing lawful speech, curtail their ability to be anonymous, and jeopardize the data security and privacy of all internet users.
