Yahoo Secretly Built Software To Scan All Emails Under Pressure From NSA Or FBI

from the uh-wait-a-second dept

So Reuters had a big exclusive report this morning about Yahoo creating “custom software to search all of its customers’ incoming emails for specific information” at the behest of the NSA or FBI. This was built last year — which came well after the Snowden disclosures, and after Yahoo had been revealed to have legally challenged earlier NSA dragnet attempts — and after it had rolled out end-to-end encryption on email.

Apparently, this was a decision made at the top by Marissa Mayer, and pissed off the company’s top security guy, Alex Stamos (who is awesome and a big supporter of end-to-end encryption) leading him to leave the company (and move to Facebook, where he is currently).

According to the two former employees, Yahoo Chief Executive Marissa Mayer’s decision to obey the directive roiled some senior executives and led to the June 2015 departure of Chief Information Security Officer Alex Stamos, who now holds the top security job at Facebook Inc.”Yahoo is a law abiding company, and complies with the laws of the United States,” the company said in a brief statement in response to Reuters questions about the demand. Yahoo declined any further comment.

Of course, this comes out less than a week after the NY Times had a big report on how Mayer de-prioritized security, despite having built up a great team of computer security experts called “The Paranoids.” But, Mayer apparently downplayed or blocked their efforts, leading many to go elsewhere. And now we find out that Yahoo agreed to create this special software for scanning all emails for certain phrases or keywords. Bizarrely, this new report notes that Mayer gave the task of writing this software not to the security team, but to email engineers, leaving the security team in the dark, until they discovered it, thinking it was malware:

They were also upset that Mayer and Yahoo General Counsel Ron Bell did not involve the company’s security team in the process, instead asking Yahoo’s email engineers to write a program to siphon off messages containing the character string the spies sought and store them for remote retrieval, according to the sources.

The sources said the program was discovered by Yahoo’s security team in May 2015, within weeks of its installation. The security team initially thought hackers had broken in.

When Stamos found out that Mayer had authorized the program, he resigned as chief information security officer and told his subordinates that he had been left out of a decision that hurt users’ security, the sources said. Due to a programming flaw, he told them hackers could have accessed the stored emails.

Now, there are still a number of open questions about this: chief among them if others, such as Google, Microsoft, Facebook, and Twitter were similarly compelled to create similar software. This may not be that meaningful, but the article does not say that it was a FISA Court “order” but rather a “directive” that compelled this:

The company complied with a classified U.S. government directive, scanning hundreds of millions of Yahoo Mail accounts at the behest of the National Security Agency or FBI, said two former employees and a third person apprised of the events.

The question then is what secret “directive” does the government have that allows such broad scanning? The most likely (but certainly not the only) possibility is a stretched interpretation of Section 702 of the FISA Amendments Act. That Section is responsible for two known programs for the NSA to collect info: PRISM, which had big tech companies sharing specific information with the NSA, and “upstream” collection in which broadband providers like AT&T would scan all traffic for certain information. Without more detail, it’s a little difficult to know what happened here, but it sounds like something in between PRISM and upstream — in which online service providers were similarly asked to scan all content for certain information.

It seems clear that Yahoo either didn’t think it could win a legal fight over this (certainly a possibility), or that it just didn’t want to. At the very least, this seems like yet another example of totally secretive rulemaking by the US government on what surveillance capabilities are legal, without any public review or adversarial process designed to make sure that civil liberties are protected. I know that many of the more paranoid folks out there think that the NSA already had deals with the big companies to scan all content, but they weren’t supposed to, and as far as we knew they did not as of a few years ago. But if that changed last year, that’s a big, big deal, and much more information needs to become public on this.

Filed Under: , , , , , , , ,
Companies: yahoo

Rate this comment as insightful
Rate this comment as funny
You have rated this comment as insightful
You have rated this comment as funny
Flag this comment as abusive/trolling/spam
You have flagged this comment
The first word has already been claimed
The last word has already been claimed
Insightful Lightbulb icon Funny Laughing icon Abusive/trolling/spam Flag icon Insightful badge Lightbulb icon Funny badge Laughing icon Comments icon

Comments on “Yahoo Secretly Built Software To Scan All Emails Under Pressure From NSA Or FBI”

Subscribe: RSS Leave a comment
TruthHurts (profile) says:

Yahoo committed treason against this nation.

Obviously Yahoo is *NOT* a law abiding company as they broke the 4th Ammendment in such a way that constitutes treason to the American people.

How does it constitute treason? It failed to protect the American people in their “right to be secure in their persons, houses, papers, and *effects*, against unreasonable searches and seizures” This constitutes grave damage to the people of this nation by Yahoo’s betrayal of trust, which is one of the definitions for Treason.

Now, I don’t know about you, but I damned sure count my e-mails as “personal effects”, and am damned sure that I have not, and will not allow anyone from the lowliest beggar to the PoTUS or SCoTUS to redefine them in any way.

Anonymous Coward says:

Re: Yahoo committed treason against this nation.

breaking the 4th has NOTHING to do with treason. The term “Treason” has a definition explicitly established in the Constitution to prevent people like you from misusing the Term.

That being said, Treason is definitely occurring just not in the manor of which you are accusing. Yahoo is not required to honor the 4th in regards to the email accounts it hosts, it is not an extension of the Law, this is just more 3rd party doctrine being used for government to skirt the 4th. Since these actions are not being used to aid an enemy of the US (that we know of) then it cannot be classified as treason. For now, it is just fucking unconstitutional… not that any of my fellow citizens give a flying fuck though. As long as they have their Cheeze Whiz and Superbowl Sundays… they are are kept little arm chair patriots.

Every Nation gets the Government it Deserves!
~Joseph De Maistre

David says:

Re: Yahoo committed treason against this nation.

The 4th amendment restricts the power of the government. Last time I looked, Yahoo was not part of the government. The 4th Amendment should have enabled Yahoo to state “Sorry folks, you are overstepping your authority unless you can show us a warrant issued upon probable cause for all of our customers and we aren’t required to be an asset to your attempt of violating the Bill of Rights.”, not required them to do so.

But apparently the Constitution is no longer the highest law of the land and superseded by secret laws that are not even open to democratic scrutiny. Nobody knows what rights and recourse Yahoo or U.S. citizens have under those secret laws: the government does what it wants to and labels its own rules a state secret. Basically everybody pretends that there has been a military coup and the Constitution has been suspended, with the Bill of Rights being optional for the government.

Yahoo is not treasonous but incompetent and sleazy if they don’t do their part in protecting their users’ rights.

The treason is committed by the government. Execute everybody involved with putting the Constitution out of order and see whether the rest then understands that swearing an oath on the Constitution is serious business.

Oaths have meanings. And it’s not Yahoo who had to swear an oath on the Constitution in order to be admitted into service of the People of the United States of America.

Justme says:

Re: Re: Yahoo committed treason against this nation.

I can’t say it’s treason, but if it’s yahoo that is running the system to scan and select targeted emails before handing them over to the nsa/fbi. Then they are most certainly acting as a agent of the government.

Which would change the equation legally, compared to the government doing the scanning and selection of emails.

anti-antidirt says:

Re: Yahoo committed treason against this nation.

The Fourth Amendment doesn’t protect you from corporation’s searches and seizures. That’s considered a given, because obviously, Yahoo can’t obtain a Warrant. Yahoo can’t set up checkpoints either (in case you were wondering). The Amendments are limitations the Government have in relation to the People. Furthermore, if Yahoo had ONLY one customer and did this, immediately you’d jump to the treason charge?

Yahoo didn’t damage the nation with it’s “betrayal of trust.” They were dipshits years ago. ANYONE using Yahoo in 2016 is someone that can’t be trusted anyway.

Not treason.

Ninja (profile) says:

As if Yahoo needed any more nail in its coffin. And it will spill in other companies as the article notes. The US Govt via their intel are dismantling any and all trust people had on their companies. One has to wonder how much it has already cost. In the end, no terrorist has ever done as much damage as the Govt itself did to the country be it by eroding Constitutional rights or directly by driving people away from doing business with the US.

I.T. Guy says:

“I know that many of the more paranoid folks out there think that the NSA already had deals with the big companies to scan all content”
[Raises hand]

Once a “conspiracy theory” pondering, turns out to be true… AGAIN!!! And again and again and again.

Ok smart people… why would the gov do this to Yahoo and not MicroGoogleBook?

Thad (user link) says:

Re: Re: Re: Re:

But MS has fought back publicly.

Now, that involved data stored on foreign servers, and it’s certainly not a guarantee that MS has never complied with any other government requests for data. But it does imply that MS’s leadership is cognizant of the importance of maintaining overseas customers’ faith that the US government wasn’t spying on their E-Mails, in a way that Yahoo wasn’t.

Still and all, E-Mail is based on outdated, unsecure protocols, and, whether companies are complying with government spying requests or not, people should always assume that nothing they put in their E-Mail is really private.

Anonymous Coward says:

“There is a lot of talk about data coming out of facebook: is it coming to me? is it coming to him? is it coming to them? They want you to think that the threat is data coming out. You should know that the threat is code going in.” – Eben Moglen

Social media and communications platform have become defacto platforms of societal surveillance.

Anonymous Coward says:

Re: Re:

“So you are looking for classes of people. You don’t know their names, but you know what they are like you know who is recrutable for you as an agent you know who are likely sources, you can give the social characteristics of your adversaries, and once you know your adversaries, you can find the influencables.

So what you want to do is run code inside facebook. It will help you find the people that you want it will show you the people whose behavior and whose social circles tell you that they are what you want by way of agent, sources what their adversaries are and who you can torture to get to them.

So you don’t want data out of facebook the day you have data out of facebook it is dead. You want to put code into facebook and run it there and get the results you want to cooperate.”

Anonymous Coward says:

Re: Re:

“Social media and communications platform have become defacto platforms of societal surveillance.”


“Social network” is simply another name for “Surveillance network”.

When Google, Facebook, Twitter, Linked-In, etc., build surveillance platforms to sell consumer data to advertisers, why wouldn’t the government use the third party doctrine to get the information?

The government could simply have purchased access to the information like any other paying customer, but by using secret rubber-stamp FISA orders, they get the information for free.

OldGeezer (profile) says:

This would only catch really dumb terrorists

Any terrorists who don’t use encryption would have to be incredibly ignorant to spell out their plots using the keywords these programs are looking for. It really isn’t hard to figure out what words and phrases would be on the naughty list. Don’t you think they would at least use some sort of code? Criminals have been doing this for ages from Mafia families to street level drug dealers. What a colossal waste of time and money.

Anonymous Coward says:

Re: This would only catch really dumb terrorists

What a colossal waste of time and money.

Not if your intent is to preserve the political status quo, and nip protest movements in the bud by identifying people to put on the no-fly list or otherwise hinder the potential leaders ability tom meet with each other, or get to Washington to lead the Protest. Asset seizure of vehicles is another handy tool for this purpose

OldGeezer (profile) says:

Re: Re: This would only catch really dumb terrorists

Agreed, but terrorism is always what they to justify any snooping. Targeted surveillance has worked in a few cases but most of the time the feds still blow it. Before 9/11 they were wiretapping the house that was used to relay messages between Al-Qaeda leaders and operatives. The NSA knew that known terrorists made it into the country with legal visas. They not only failed to share intelligence with the FBI but they somehow lost track of them much of the time. A flight instructor informed authorities of mid eastern men who wanted to fly but did not want to learn to land a plane. The no fly list is such a joke that these men with known terrorist ties were allowed to board using their real names. Many other mistakes were made that could have prevented the attack. The feds have the mentality of “collect it all” that has NEVER worked.

Padpaw (profile) says:

Re: This would only catch really dumb terrorists

not meant to catch terrorists its meant to catch dissenters.

Political protesters, people with jobs those in charge don’t like, people that do legal stuff those in charge don’t like.

Whoever is running the US government has no respect for their own laws or constitutional rights. American citizens do not have any protections they think they do, when those running the show ignore them.

That One Guy (profile) says:

Worse than nothing

This was built last year — which came well after the Snowden disclosures, and after Yahoo had been revealed to have legally challenged earlier NSA dragnet attempts — and after it had rolled out end to end encryption on email.

So they spend time and money to implement end-to-end encryption for the email, and then spend time and money to implement code that makes that completely worthless.

At this point they’d have been better off never bothering with end-to-end encryption at all because it’s blatantly clear that they’ll break it the second someone with enough power asks them to.

afn29129 (profile) says:

Re: Worse than nothing

“and after Yahoo had been revealed to have legally challenged earlier NSA dragnet attempts — and after it had rolled out end-to-end encryption on email. ” What Yahoo has ISN’T end-to-end encryption on/for email. End-to-end is where the sender or recipient have the keys, and nothing to do with a transit-provider. If Yahoo really is making that claim, then they are lying their asses off.

Anonymous Coward says:

Re: Worse than nothing

To be fair, there’s more than one government in the world that has the technological capability to split and mirror data from fibre optic cables. Those governments theoretically would be locked out still.

But as you pointed out, it looks like the NSA is trying to replicate BULLRUN domestically, which is a gigantic “fuck you” to Silicon Valley. We can reasonably assume the same has been imposed on other tech companies that publicly stated that they encrypted data between all their datacenters – Google and Facebook included. Perhaps this is the FBI order Sergey Brin and Larry Page refused to comply with in 2014 and were subsequently arrested for and walked out of Google HQ in handcuffs. Obviously they have since capitulated and agreed to comply with the order from the FBI since.

Anonymous Coward says:

Re: Re:

“Wouldn’t it be sweet sweet irony if the 500 million users info that was copied was caused by the writing of a special program to search all customers’ emails in transit.”

Except that the 500 million appeared to include very old accounts that had not been used in aeons. (I read somewhere) that was part of Marissa Mayer’s justification for not doing an automatic password reset or sending out an earlier notification – the fear that users with old accounts would close the account rather than create a new password, hence leading to LOSS OF REGISTERED USERS – a disaster far worse PR than, you know, letting personal information be stolen. Far far worse, when one has stock prices, your own reputation and the next gig to care about.

kallethen says:

Re: Re:

My email address from my old dial-up ISP (AT&T) is managed through Yahoo Mail. I would be surprised that’s a wide-spread userbase of Yahoo Mail.

Between this and other recent revelations, I’m seriously thinking I should completely shut down the account and use a new email address. The problem is… who to trust?

Padpaw (profile) says:

Re: Re: Re: Re:

In my case I am not all that worried to be honest. I don’t use email for my personal life just for the various fake personalities I have set up for differant online things.

Not a single bit of my real name or life is associated in any way with any of my emails.

But then again I have always been paranoid about such things.

UniKyrn (profile) says:

So who watched the NOVA episode about 15 years of terrorism and was annoyed when it wasn’t a documentary about the abuses of .gov, it was a puff piece about security theater?

We don’t have the ability to stop them because we don’t know who they are, what they’re doing and we’re denied legal means of fighting back. Some might call it searching for a needle in a haystack. These days, it seems more like finding the hay in a stack of needles.

HegemonicDistortion says:

Even worse than previous programs

The biggest development here, it seems to me, is that this operation completely does away with any notion of a criminal predicate. Previous efforts have mostly been about “connecting dots,” i.e. searching those communicate with suspected terrorists, or at least those communicating with people from what the US deems TerrorLand.

But this, this is a search of everyone, right from the start, no suspicion or even mere connection to some suspect. It’s the equivalent of searching everyone’s house for evidence they committed some crime. It’s an actual general warrant.

That One Guy (profile) says:

Re: Even worse than previous programs

Nonsense, it was a very narrowly worded ‘request’, it only applied to one entire company. The government could have ‘requested’ that Yahoo perform the same scanning of the databases of every US-based company, which of course would have still been very narrowly worded because it only applied to the companies based in one country.

And when, a few years down the road they ‘request’ that every company that sells to or offers service to anyone in the US does the same thing it will still be a ‘narrowly worded request’, because it only applies to companies on one planet.

Anonymous Coward says:

Wait a minute...they BUILT software?!

And now we find out that Yahoo agreed to create this special software for scanning all emails for certain phrases or keywords.

Why would anyone who knows ANYTHING about email do that?

Software to do exactly that has existed, in numerous forms, since the last century. To name just one piece of it, out of dozens and dozens: SpamAssassin. (Of course SpamAssassin does much more than that, but stay with me here.)

It would be the work of a few hours to take the list of words, character strings, or phrases provided by the government, configure it into SA, remove everything else, and set it up to either copy messages to a secondary mailbox or divert them entirely (so that the intended recipient never got them). I’d imagine that the former would be desirable in order to avoid alerting recipients and thus, eventually, senders.

This is a trivial task for anyone who’s run a mail system for a couple of years and has worked with the various moving parts, i.e., SMTP servers, POP and IMAP servers, anti-spam configuration, and so on. It’s not a development task: it’s a configuration task, since all the pieces already exist and just need to be put together.

Incidentally, while not in play here, searching already-stored mail is equally trivial. See “grepmail” for — again — one of many readily-available tools.

So why is Yahoo telling us that they had to build this? And especially why are they telling us that when we all know they already have content-scanning software deployed in the their mail system? (It’s part of the anti-spam, anti-malware defenses they like to brag about.)

Anonymous Coward says:

Re: Wait a minute...they BUILT software?!

“So why is Yahoo telling us that they had to build this? And especially why are they telling us that when we all know they already have content-scanning software deployed in the their mail system? (It’s part of the anti-spam, anti-malware defenses they like to brag about.)”

They deliberately did NOT WANT their own IT/IT security teams to know what they were doing so they had this done by the email code team. The IT security team discovered it after it had gone live and they reported it upwards because they thought they’d caught Y! being hacked from outside.

Skeeter says:

Missing the Details

Funny, everyone focused on the actions we know are already transpiring, not one-single-question about ‘what string or phrase were they searching all these e-mails for’.

Isn’t it quite curious that three sources dropped-a-dime on the government, Yahoo, and what happened – but across a dozen stories, not one single one stated ‘what they are searching for’?

HINT: It is ONE person (and goofy search strings they think will lead them to him by querying), and beyond that, I can’t say more. Maybe someone could share more-interest in ‘what’ they are searching for, instead of the obvious – their breaking the law to find him, er, it.

Anonymous Coward says:

Skeeter, you can’t seriously expect us to believe that only one person is being looked for with this. Even in the extraordinarily unlikely case it is only one person, it’s not like the FBI and/or NSA will close up this capability and never use it again.

I tire of spooks who pretend that the arbitrary discretion of Intelligence Community policymakers and analysts somehow make this okay in the slightest. It’s a grossly disingenuous argument by someone who doesn’t have the courage to stand by their convictions.

John Mayor says:


Afterupon learning a couple weeks ago of the hack of ID on a 1/2 BILLION Yahoo customers, one should wonder how this company is managing to stay solvent!… and!… why users haven’t kicked in the front doors of Yahoo’s headquarters (let alone, haven’t mounted the largest civil law suit in US history, and the largest “criminal roundup”, and prosecution in US history!)!
This CYBER SHOCK AND AWE, has– I feel!– caused Netizens to become SHELL SHOCKED! I mean… one week you’re wondering if an “intergalactic invasion” has hit earth, and the next week you’re wondering whether the “aliens” are actually living next door!… and whether Donald Sutherland will knock any minute, and ask that your children accompany him to a “landing pad”!
It’s A-B-U-N-D-A-N-T-L-Y C-L-E-A-R– to me!– that the BEHAVIOR of our “Internet Gatekeepers” is affecting/ impacting on everything we’re attempting to do on the Net! In attempts to “catch the bad guy”, these “Gatekeepers” have moved into the realm of P-S-Y-C-H-O-P-A-T-H-I-C B-E-H-A-V-I-O-R!… and leaving many Netizens with little– OR N-O!– recourse!
It’s time for a G-L-O-B-A-L C-O-A-L-I-T-I-O-N O-F N-G-O-+-N-P-O S-E-C-U-R-I-T-Y A-D-V-O-C-A-C-I-E-S, T-O M-O-U-N-T T-H-E L-A-R-G-E-S-T N-E-T-I-Z-E-N L-A-W-S-U-I-T I-N H-I-S-T-O-R-Y, A-N-D T-H-E E-S-T-A-B-L-I-S-H-M-E-N-T O-F T-H-E L-A-R-G-E-S-T “N-E-T-I-Z-E-N I-C-T D-E-F-E-N-C-E L-E-A-G-U-E I-N H-I-S-T-O-R-Y”!
Enough!… of allowing Netizens to walk around with their hands over their ears!… to lesson the impact of the “daily explosions”!
Please!… no emails!

Anonymous Coward says:


I know your trolling, but when chunk of your text is big and dashed you just make it harder to read instead of adding emphasis like you think.

Also the shouting just makes people not want to read the post.

If you just share your insight like a normal person more eyes will bother to read it. ;P

David says:


to me the denials from other companies sound like “well yeah, we obey the same kinds of request, just less amateurishly”.

It seems to take a Mayer to effectively say “You want a backdoor? I’ll give you a backdoor! Let’s pull down our city walls for you!”

She puts the ancient Trojans to shame. It would seem that she left her “Don’t Do Evil” motto at Google. Not that they had any use for it.

Lord Lidl of Cheem (profile) says:

Having read about this earlier and then watching the google livestream last night all I could think of was – what if google are doing the same thing and prepping a government version of its assistant…

OK Google, tell me what crimes are happening in this area?
“There are 18 drug deals currently going occurring in a 15 mile radius – would you like to see them on a map?”

Mac (profile) says:

Problems with PRISM? Use PrismCipher!

Nobody is going to stop the NSA from mining, recording, indexing and analyzing email traffic. But we can make the effort cost a little more with encryption. PrismCipher is user-to-user encryption that works with Gmail and Yahoo email. It prevents Google, Yahoo, the NSA and anyone else from parsing and analyzing your emails.

Add Your Comment

Your email address will not be published. Required fields are marked *

Have a Techdirt Account? Sign in now. Want one? Register here

Comment Options:

Make this the or (get credits or sign in to see balance) what's this?

What's this?

Techdirt community members with Techdirt Credits can spotlight a comment as either the "First Word" or "Last Word" on a particular comment thread. Credits can be purchased at the Techdirt Insider Shop »

Follow Techdirt

Techdirt Daily Newsletter

Techdirt Deals
Techdirt Insider Discord
The latest chatter on the Techdirt Insider Discord channel...