Cricket Revealed As Mobile ISP That Was Blocking Encrypted Emails
from the nicely-done dept
A few weeks ago, we wrote about how VPN company Golden Frog had quietly revealed in an FCC filing that an unnamed mobile broadband provider had been (even more) quietly blocking people from sending encrypted emails — basically blocking users from making use of STARTTLS encryption. The Washington Post has now revealed that the mobile operator in question was Cricket — a subsidiary of AT&T, and that it stopped blocking such encryption a few days after our post was published.
Cricket did not address repeated questions about the issue and did not alert customers, many of whom rely on Cricket as their sole Internet service, that they would not be able to protect their e-mails from prying eyes. AT&T, which absorbed Cricket when it acquired Leap Wireless last spring, did not respond to a request for comment.
Cricket said in a statement to The Post that it “is continuing to investigate the issue but does not intentionally prevent customers from sending encrypted emails.”
The issue appears to be that Cricket started using some Cisco firewall equipment to block sending encrypted emails through Port 25. It’s true that many ISPs block Port 25 entirely, as it’s often used for spamming. What Cricket did here was to just try to block encrypted emails over Port 25, which in some ways is being more permissive than other providers who block it entirely. Yet, still, the way it did so was somewhat misleading and still concerning. While the intent here may have been reasonable, any time you have an ISP stepping in and quietly making the decision itself to block encrypted traffic while allowing other traffic it should raise questions about the security for end users. Yes, there’s a constant battle against spam, but there may be better ways to deal with it than single-handedly blocking email encryption.
Filed Under: blocking, encryption, fcc, net neutrality, open internet, port 25, spam
Companies: cricket, golden frog
Comments on “Cricket Revealed As Mobile ISP That Was Blocking Encrypted Emails”
Well, duh...
Cricket’s prepaid. Burner phone users *have* to be up to something.
Cisco ASA firewalls.
“inspect esmtp” is the default setting for Cisco ASA Firewalls across at least the 7.x, 8.x, and 9.x code trains, and causes exactly this behavior. It’s a single line in what is generally a very large config file, buried near to the end of the config, trivial to overlook, and generally a pain in the ass.
To make things even better, “inspect esmtp”‘s functionality is further obfuscated by the fact that most “inspect xyz” commands on the ASA actually allow for proper handling/operation of protocols that require special treatment – packet manipulation to deal with nat/pat, firewall pinholing, etc. Examples of protocols requiring inspection for proper operation on the ASA platform include, but are not limited to, netbios, sunrpc, sip, h323, etc.
Nothing about “inspect esmtp” _or_ it’s location in the ASA configuration file implies “break critical esmtp functionality”
Unless/until you’ve been bitten by this, most people firewall administrators don’t know to look for it, and the reaction I’ve seen from most people when they find the solution is along the lines of “damnit cisco…”
Frankly, having deployed a fair number of Cisco ASA’s myself, this sounds more like a missed configuration setting followed by an “oh crap” moment on a new deployment than a malicious “let’s break encrypted email” conspiracy.
Re: Cisco ASA firewalls.
I’ve marked this “insightful” and would do so a dozen more times if the interface permitted. It’s almost certain that this is the root cause of the issue and that it has nothing to with net neutrality, anti-spam methods, defeating encryption or anything else.
As you say, lots of people have been bitten by this, going back to the days of the Cisco PIX, because the documentation doesn’t make it clear and so the only reliable way to find out that one has picked the wrong setting is by (painful) experience. If the ISP was really trying to block encrypted email submission from clients to servers, then they’d be doing this on port 465, not 25.
Re: Re: Cisco ASA firewalls.
Rich,
Thanks for the note. I was fairly certain this was true on the first PIX 525’s (running 6.x code) I worked with, but that was more than a decade ago, and I couldn’t remember for certain. I don’t go back further than 6.x on the PIX’s, and I inherited that one, so I didn’t have the opportunity to be bitten (by this item) on its initial deployment. 🙂
Also, I feel I should point out: If Cricket/ATT really wanted to break ESMTP w/ an ASA – this was an exceptionally ham-fisted way to do it. There are far subtler ways you could do this with an ASA, but frankly, it wouldn’t be my pick for the job. A company like Cricket/ATT can afford more effective mechanisms.
Re: Cisco ASA firewalls.
Ditto on “insightful”, and a well-written summary. However, this incident is also a great illustration where the “act casual, say nothing” response so many companies embrace only breeds suspicion and distrust. How I wish more companies would stop being run by their legal teams (or hire a better ones), and be more candid when they make a mistake.
Wouldn’t something like this have worked better?
“We found an obscure and confusing setting in a new piece of network equipment that was causing problems with esmtp (encrypted e-mail). Managing or blocking smtp/esmtp can be useful in spam reduction, and we really hate spam. However, what would seem to be an intuitively obvious choice (and the system default) for this particular setting turned out to be the opposite of what we wanted. Our intention was never to block legitimate encrypted mail. The problem has been corrected and administrator of that equipment is writing, “I will never make this mistake again” one thousand times on a smart board. We want to thank everyone who brought this to our attention.”
Re: Re: Cisco ASA firewalls.
Question is: Why is this a default setting for a router?
Re: Re: Re: Cisco ASA firewalls.
A Cisco ASA isn’t a router, it’s a firewall.
Generally speaking, routers and firewalls have very different design philosophies: I have to specifically configure my routers to block traffic. I have to specifically configure my firewalls to permit traffic.
That said, the relative location of the “inspect esmtp”
setting coupled with the lack of clear documentation is still a problem.
Re: Re: Re:2 I have to specifically configure my firewalls to permit traffic.
Except this firewall wasn’t blocking SMTP traffic, only encrypted SMTP traffic.
Which is a very peculiar setting for a default, don’t you think?
Re: Re: Re:3 I have to specifically configure my firewalls to permit traffic.
Did you not see the ‘inspect esmtp’ part of sig’s post?
Re: Re: Re:4 I have to specifically configure my firewalls to permit traffic.
The “e” in “esmtp” stands for “extended”, not “encrypted”.
https://en.wikipedia.org/wiki/Extended_SMTP
Re: Re: Re: Cisco ASA firewalls.
Or: Why is AT&T staffing of competent people wholly lacking?
Re: Re: Re: Cisco ASA firewalls.
As has astutely been pointed out, it’s a firewall, not a router.
Part of the blame here can be laid at Cisco’s feet: they have to have known, given the copious discussion of this online that extends over a decade, that the option is confusing and poorly documented. Adding explicit clarification including examples and caveats would probably help engineers avoid making this mistake — and I’ve no doubt that it’s in place in many other operations besides this one.
One of the other things in play is that not everyone who configures firewalls (or routers) is an expert in application protocols like SMTP (or HTTP or FTP or SSH or anything else). They probably have a working knowledge of these things, but don’t have the SMTP chops that a long-time mail system admin has or the HTTP savvy that a long time web admin has. If they configure and test basic functionality, they may conclude that it’s working — and after a fashion, it IS working. It’s just not working correctly.
I don’t think this was at all a deliberate choice. It’s the kind of mistake that we all make all the time, and were it not for the push over the last 2-ish years toward encrypted services, it might not have even been noticed. (And it probably WASN’T noticed when it was first put in place.) We’re going to see a lot more of these as users get better at using encryption and at diagnosing failure points. Some of those might turn out to be deliberate obstruction, but I don’t think this one is. And in terms of netfuckery, we have far more important fish to fry, like Verizon’s forced HTTP headers and Comcast’s Javascript injection, both of which quite clearly ARE deliberate and — as far I know as the moment — unexplained.
Re: Re: Re:2 Cisco ASA firewalls.--- SMTP is Slightly Miscast.
I think there is a case for adding a capacity to the POP3 protocol (port 110 or 995), to allow uploading e-mails to the e-mail server. The client would have to be logged in, the e-mail would have to be sent from the account the client was logged into, and the e-mail server would automatically check the FROM and RETURN-PATH fields. This would mean that it would be much easier at the port level to distinguish between legitimate and illegitimate outgoing e-mail traffic. Under this system, the client has a secure connection to one or more e-mail servers, and what those e-mail servers do with outgoing e-mails is their responsibility, not that of the ISP. The same logic would apply to IMAP, of course, and the APPEND/CATENATE extensions seem to be a step in the right direction.
http://en.wikipedia.org/wiki/Post_Office_Protocol
http://en.wikipedia.org/wiki/Internet_Message_Access_Protocol
http://tools.ietf.org/html/rfc4469
SMTP would be restricted to the purpose of moving mail from one mail server to another, and firewalling rules could be developed around that function.
Re: Re: Cisco ASA firewalls.
Isn’t this EXACTLY what Google did when they were caught packet-sniffing?
And their in-depth comments were used against them in Europe as an admission that they knew they were doing this.
http://www.wired.com/2010/06/google-wifi-sniffing/
“Lawyers suing Google claimed Thursday they have discovered evidence in a patent application that Google deliberately programmed its Street View cars to collect private data from open Wi-Fi networks, despite claims to the contrary.”
I remember that the freeware code they were using was discussed on here with people pointing out that it logs data that could include usernames and passwords if they were sent with no encryption.
So that’s why they don’t do it.
Re: Re: Cisco ASA firewalls.
I’m not affiliated with anyone involved, btw.
I think the short answer is, they don’t/won’t issue a statement because there’s effectively no upside. Best case, they issue a heartfelt press release, people accept it at face value, and 3 weeks later no one remembers it happened. If you don’t end up with best case, though, you get to take your pick of headlines:
Use your imagination. The Reporters, bloggers, and conspiracy theorists certainly will. No statement might be annoying, but at least it doesn’t allow your own statement to be twisted and used to vilify you..
I wonder if this is why my ISP blocks STARTTLS (they support smtp over ssl though, but it was a pain to configure exim4 to support this, even with someone else’s helpful guide specifically for the same ISP).
Earthlink promised to encrypt by June 2014
but nothing has been done so far.
Does anyone here have an email for an Earthlink executive so that Earthlink can be shamed into following through on their promise?
I’m not so worried about the NSA, but security at wifi hotspots is non-existent.
That explanation makes no technical sence, the email should be scanned by the smtp server. And blocking port 25 is useless because email can be sent via any port. 25 just happened to be easy to remember in the early days of the net.
Wait… Who uses AUTHTLS on port 25?
This looks like a client problem more than anything to me.
Question is: Why is this a default setting for a router?
Thanks for posting this techdirt. The original article got excoriated elsewhere, with people commenting on your naivete at one end of the spectrum and others commenting on your outright shrill incompetence at the other.
Let this be a lesson that networking is a complicated beast, and it’s wise to get the full story before going off on uninformed rants about potential intentional and scary encryption stripping.
Last year (2014) i also had my emails hacked and i have been looking into and i have been looking into email encryption softwares. I found some good information from this website http://www.hackedemails.com/email-encryption/ and hope that it might be useful for you also.