Is It Your Fault If Someone Else's Spam Filter Catches Your Important Email?

from the playing-the-blame-game dept

We've all heard the stories about overeager spam filters catching messages they shouldn't -- but now there's a case on hand that ended up costing a school district a quarter of a million dollars by not awarding a phone contract to the cheapest bidder, because a spam filter ate an important email. However, where the story gets even more interesting is in the details, as people start figuring out who to blame. Normally, you'd say that anyone making a bid on such a big contract should know not to trust that an email had arrived safely. However, this email wasn't the original bid (which was delivered in a hard copy), but a followup question from the district which clearly stated: "We must have our response to this request returned via e-mail ... in order for your company's response to be considered further." After all that, though, the school district is still blaming the vendor, saying a good bidder knows it needs to outwit spam filters.

Reader Comments

Subscribe: RSS

View by: Time | Thread


  • identicon
    Brad, 24 May 2006 @ 2:39am

    Hah!

    As long as we can blame anyone but ourselves, we're safe.

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 24 May 2006 @ 3:23am

    why is it so hard for people to hold up there hands and admit that they were wrong or made a mistake? things would go a lot smoother if they did

    or am i living in a fantasy world there?

    reply to this | link to this | view in chronology ]

    • identicon
      Skippyboy, 24 May 2006 @ 6:15am

      Re: Anonymous Coward Fantasy World

      Boss! Boss! de Plane! de Plane!

      Does that answer your question? :P

      reply to this | link to this | view in chronology ]

    • identicon
      Tom, 24 May 2006 @ 8:03am

      Re: Because admitting fault is akin to a guilty pl

      If you hold up your hand and say "My fault, sorry" you have just admitted liability for which you will pay. Who wants to volunteeer if you can wait for the courts to decide that you really are at fault; they may decide you aren't after all. . It is a fantasy to believe that as an adult you can just admit you're wrong and say you're sorry and then nothing will happen. that may work with friends, but not most everywhere else.

      reply to this | link to this | view in chronology ]

  • identicon
    ?, 24 May 2006 @ 3:45am

    Sounds like the people in charge of education

    Have some learning to do themselves.

    reply to this | link to this | view in chronology ]

  • icon
    Peet McKimmie (profile), 24 May 2006 @ 4:43am

    Can someone clarify, please...?

    Was it the school district's follow-up question that was eaten by the vendor's SPAM filter, or was it the vendor's reply that was eaten by the school district's SPAM filter? I can't work it out from the article above...

    reply to this | link to this | view in chronology ]

    • identicon
      me, 24 May 2006 @ 6:29am

      Re: Can someone clarify, please...?

      um... how could the vendor reply to an email that they never received? It's really not that hard to deduce on your own.

      reply to this | link to this | view in chronology ]

      • icon
        Peet McKimmie (profile), 1 Jun 2006 @ 7:48am

        Re: Re: Can someone clarify, please...?

        "It's really not that hard to deduce on your own."

        From what it says up above, the school sent a message saying that the vendor had to reply by email. It also says that an unspecified message was eaten by a SPAM filter. It doesn't say whether the message that was eaten was the school's request for a reply or the vendor's reply, and it doesn't actually say that the vendor replied.

        In summary, yes, it really is that hard to deduce on your own without any further information, which your "helpful" reply doesn't contain.

        reply to this | link to this | view in chronology ]

    • identicon
      Dan, 24 May 2006 @ 7:48am

      Re: Can someone clarify, please...?

      It was the Vendor's reply that was eaten by the school district's spam filter.

      And yet, it's not their fault... ;)

      reply to this | link to this | view in chronology ]

      • identicon
        ZA, 24 May 2006 @ 11:27am

        Re: Re: Can someone clarify, please...?

        I sure hope someone lost their job over losing, what must have been a multi-million dollar contract. How difficult is it for a sales person to follow up with a phone call? "Hey, did you receive the email information that you requested?"

        "No? Really? I know it was sent. Let me try to find out why you didn't receive it."

        Thank you for playing; have a nice life!

        reply to this | link to this | view in chronology ]

  • identicon
    William Lane, 24 May 2006 @ 4:46am

    Spam Filter

    Why would ANYONE send an important email and not have a confirmation sent back to let the sender know his email has been opened. If the confirmation was not returned in a timely manner or received, then the sender could take action to find out if they received it.

    reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 25 May 2006 @ 10:27am

      Re: Spam Filter

      How do you "have" a confirmation sent back when you don't run the receiving system? You can "request" confirmation, but that is far different from commanding it. And, by the way, most people know that receipt confirmation requests are almost always universally disregarded anyway.

      reply to this | link to this | view in chronology ]

  • identicon
    lar3ry, 24 May 2006 @ 5:21am

    No such thing as a confirmation

    The Entire World Doesn't Use Outlook.

    Just because Outlook and a few other mailers support delivery status notification (DSN), there is no guarantee that anybody that you send email will support it. There is no guarantee of delivery of email--it can be rejected (you'll get notified), or it will be silently discarded like the SPAM filters of some ISPs.

    The relevant email protocols allow for delivery failure notification (in other words, "don't bother us, spammer!") to inform the sender that the email wasn't received. However, it's very simple to configure most mail transfer agents to silently drop messages. I think the reasoning is that SPAM affects network traffic (there's so much of it) that sending out failure notices increases the network traffic, so you combat SPAM by simply ignoring it.

    I had an interesting thing happen about 11 years ago. Somebody tried to send an email with the name of a wine in it. Parental filters (not mine, but the sender's) kept on refusing to deliver the email, saying the message was inappropriate. A LOT of digging turned out that the word "Chardonnay" has a "bad word" in character positions 2-7.

    Back to the main point, how can the vendor know if his message was tagged as SPAM if he didn't get a message indicating failure of delivery (again, this is not the same as "confirmation of delivery")?

    The onus should be on ISPs to simply route email properly--or send back a failure stating why the message wasn't delivered. However, the ISPs are acting much like the old Bell System of the 1970s, which Lily Tomlin lampooned as Ernestine the Telephone Operator: "We're the phone company. We don't care. We don't have to."

    Since this cost the vendor money, I think the company that lost out has a good case against the ISP or maintainer of the SPAM filter. (Oh, my... am I advocating another lawsuit???)

    reply to this | link to this | view in chronology ]

    • identicon
      Mikhail, 24 May 2006 @ 6:47am

      Re: No such thing as a confirmation

      Correct, the entire world does not use outlook. However, the entire world of email users is capable of typing "Please send me a reply to this email confirming that you have received it." in their email's text.

      If you don't receive the reply, you can simply send another email - if it was the case where the School District was unwilling to take phone calls and would get irritated by receiving multiple emails, then they weren't looking for the lowest bidder to begin with (and why should they?).

      reply to this | link to this | view in chronology ]

    • identicon
      mthorn, 24 May 2006 @ 10:10am

      Re: No such thing as a confirmation

      I worked on a spam filter system for businesss where the message was silently delete, AFTER being stored in web friendly interface for retrieving spam.

      I think ISP should be required to store spam for at least 30 days in a location its users can access.

      However, just like regular mail, you get no guarantee of the mail being recieved unless you require delivery notification or a signature. Similar services exist for email and should be used for important emails like this. Even without spam filters, email delivery is not guaranteed. Servers, proxies, mail programs, etc can all fail along the way, losing the mail in the process.

      PS - The spam filter I worked on knew the difference between "hardon" and " hardon ".

      reply to this | link to this | view in chronology ]

  • identicon
    Starseed, 24 May 2006 @ 5:26am

    Here's the solution... Check your junk mail every once in a while! Not that difficult. The school district is clearly to blame here. They made a mistake. Woop! It happens, it just happened to be a pricey one.

    reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 24 May 2006 @ 8:54am

      Re: by Starseed

      I worked at a school division as a central administrator. The SPAM filter caught about 16,000 emails a day. If you want to go through all of them, go right ahead, but it is far too time consuming to go through the JUNK MAIL folder every once in a while, as you suggest.

      Often, if someone was expecting an email from someone, and it didn't come through, it was a fairly simple job to go through the filter and find the email (because we knew who it was from), and add that sender to the safe list.

      However, if the school was not expecting the email, they would never have known to go in and check for it.

      reply to this | link to this | view in chronology ]

      • identicon
        mthorn, 24 May 2006 @ 10:11am

        Re: Re: by Starseed

        Good spam filters are able to give a degree of certainty that a mail is spam. High certainty can be ignored, lowe certainty should be investigated. The problem is most spam filters at the ISP level just drop the message, never alerting anyone that it happened.

        reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 24 May 2006 @ 5:45am

    Blame whatever management insisted on a spam filter. Personally, I find not giving my email address out to everyone in the world, not using it on sites, and just using the delete button seems to work best, at least I don't miss anything.

    reply to this | link to this | view in chronology ]

  • identicon
    bmee, 24 May 2006 @ 5:50am

    Ask for an answer

    School district should have checked their spam messages.

    Another mistake: "We must have our response to this request returned via e-mail ... in order for your company's response to be considered further."
    I think the vendor should have required a response to both cases: want or don't want to be considered further. If they don't get a response they may insist, maybe a phone call, just to hear "oops! it was on the spam folder"

    reply to this | link to this | view in chronology ]

  • identicon
    Dave, 24 May 2006 @ 5:53am

    spam filter catches important email

    Dear teach, the spam filter ate my homework ..

    reply to this | link to this | view in chronology ]

  • identicon
    ITBubba, 24 May 2006 @ 5:57am

    Response to "No such thing as a confirmation by la

    If the vendor was blocked by the schools spam filter then all it takes is a phone call. If you pick up the phone and make a call you will be able to determine if your email was received. If you are a good salesperson then you follow up on ALL correspondence. The blame is on the vendor for not following up with the school to verify that the email was received.

    If the school was blocked by the vendors spam filter then the blame still lies with the vendor. Don't block potential clients emails.

    The customer is NEVER to blame when YOU are trying to bid on work. It comes down to how bad the vendor really wanted the contract. It looks like they didn't want it enough.

    reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 24 May 2006 @ 6:07am

      Re: Response to "No such thing as a confirmation b

      "The customer is NEVER to blame when YOU are trying to bid on work. It comes down to how bad the vendor really wanted the contract. It looks like they didn't want it enough."

      Sure they are. If you follow their instrustions on how to submit the bid, or in this case an update, and the customer drops the ball how is it the vendor's fault?

      reply to this | link to this | view in chronology ]

    • identicon
      Michael, 24 May 2006 @ 7:27am

      But How?

      "If the vendor was blocked by the schools spam filter then all it takes is a phone call. "

      HAHAHAHAHA!!! How could they call? The school was trying to get a contract for their phones!


      Michael

      reply to this | link to this | view in chronology ]

      • identicon
        *Rolling eyes*, 24 May 2006 @ 8:44pm

        Re: But How?

        There's always someone to call. If the schools actually had no phones (not just looking for a new contract when their current one expired) then there would have been a point of contact at a district office or somewhere else.

        Think before you open your dumb mouth.

        reply to this | link to this | view in chronology ]

  • identicon
    I, for one, 24 May 2006 @ 6:27am

    Mail systems

    Look at old legal case histories and you will see thousands of "I sent it. No you didn't. Yes I did..." yarns. At one time in England the Royal Mail was considered infallible. A mail coach could be impeded by no one, they carried armed guards, on recipt it became the property of the crown and obstructing the Kings Mail was a crime that carried the death penalty. Consequently an entire branch of law was built on the idea that once you placed a letter in the box it was tantamount to having been delivered to the recipient. While this was mostly for the benefit of those seeking money, it did set a useful standard in law, even if it didn't work in practice and was open to abuse. Many contract law disputes have been settled on the basis of when an invitation or response was _sent_, not when it was recieved. The evidence in those cases would have been the franking mark of the post office closest to the sender.

    To my knowledge as it stands, the law, both in the UK and the USA, is in complete disarray regarding email. Lawyers and judges have been quietly ignoring the issue hoping it will go away for a long time. I'm not even sure there is any legal basis to an electronic communication, since it is not possible to confirm or deny it, it carries no signature and has no certainty of delivery. Real, serious companies still do their business on paper, CEOs carry a pen in their jacket pocket.

    Furthermore, this state of affairs benefits many companies because of the manner in which they conduct business and their attitude to customers. They hide behind a wall of unaccountability, isolated as far as they can be from the rest of the world. Just try finding a phone number for any company these days. As such they can deny the receipt of any inconvenient communication.

    Good advice: If you ever want to really achieve something, get people jumping and asking "How high Sir?!" send a registered letter (I don't know what you call this in the US)

    reply to this | link to this | view in chronology ]

  • identicon
    SPR, 24 May 2006 @ 6:49am

    Important Email

    It is foolish to depend exclusively on email for anything important. While email is relatively reliable it can fail, and usually does, at the worst possible moment.

    reply to this | link to this | view in chronology ]

  • identicon
    Mark, 24 May 2006 @ 7:00am

    Another typical problem is lack of technical knowl

    We see more and more companies putting in spam attacking solutions and nobody knowing how it works. Even if we know a message fails - how do you get it through if there's nobody technical at the filtering side who knows how the thing works.

    And worse than just keyword filters, is those which require DNS to be a certain way - for message headers to match machine names - and things like that.

    We've had to resort to private yahoo accounts to transfer files - assuming you don't have time to snail-mail them.

    reply to this | link to this | view in chronology ]

  • identicon
    Nilt, 24 May 2006 @ 7:27am

    Ever hear of a Read Receipt?

    Yeah, sure not everyone uses Outlook. I'm pretty sure, though, anyone bidding on a wireless contract would have a mail client capable of handling read receipts.

    For crying out loud, if those things weren't intended or just this sort of thing, I don't know what they were intended for.

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 24 May 2006 @ 7:38am

    Courier Fallacy

    There's an old story about Napolean and the courier fallacy...or something like that. Goes something like...

    N sends a message to a subcommander to attack at a very specific time, in order to coordinate an attack. It is imparative that the subcommander replies that he received the message so that N doesn't attack without support. The Subcommander receives the order, confirms, and sends a reply. Now, the subcommander needs to know that N received his reply so that he knows to attack at the coordinated time. If N doesn't receive the confirmation, then the subcommander may be left to attack without support, so he needs confirmation that N received confirmation.

    And so it continues back and forth.

    The moral....if it's important (and I consider anything worth > $1000 important) talk to someone in person or over the phone. It's a bad policy of the school to rely on email. Change the policy and move on. Is the vendor entitled to any compensation...nope. It was a mistake. Not every mistake requires a lawsuit and compensation. That's how we've become the litigious society that we are.

    reply to this | link to this | view in chronology ]

  • identicon
    Charlie, 24 May 2006 @ 7:44am

    Spam Filters

    I have my MTA configured to generate a permanent delivery error when it recieves spam, and then deliver the message anyways. If the spam is coming from a custom SPAM MTA, the error is ignored. If it is a real mail server, it should generate a bounce to allow the sender to correct the error.

    It isn't rocket science.

    reply to this | link to this | view in chronology ]

  • identicon
    Tom, 24 May 2006 @ 7:47am

    The "blame game"...

    ... and of course, nobody is accountable here. Least of all the school district who is obviously held hostage by the technology that they serve.
    I am in the middle of trying to deal with a startup who wants my help, but keeps losing my emails because of yahoos spam filters. IMO, anyone who doesn't understand the infrastructure they are beholden to deserves whatever mediocrity life brings them- try running a business from a phone booth instead.

    reply to this | link to this | view in chronology ]

  • identicon
    JerseyRich, 24 May 2006 @ 7:49am

    Typical...

    Typical bureaucratic "it ain't my fault" BS.

    If you REQUIRE responses via email then you bear the burden of making sure the email gets received.

    Sure, it wasn't the district's fault, but they should acknowledge that it wasn't the businessman's either.

    The district finance director's comment about"a good businessman should follow up to make sure it was received" is a cop-out, plain and simple.

    reply to this | link to this | view in chronology ]

  • identicon
    Uncommon Sense, 24 May 2006 @ 8:30am

    What the??

    First, if you send a life-and-death email, forward a copy to self as a proof (it's kinda like certified mail), at the very least it gives you something to show as evidence.

    Second, the bidder states that he sent multiple emails yet only the most critical failed. I believe there may be more to this story than at first glance...but that's another story about unfair bidding practices.

    reply to this | link to this | view in chronology ]

  • identicon
    Bryan Price, 24 May 2006 @ 8:32am

    Bad spam filters

    I've run into this with my personal website's ISP. Their spam filter doesn't filter, exactly, it throws it away, so I don't even get a chance. He claims that there's nothing to be done about it, although I tend to doubt it. Since the ISP belongs to a personal friend, that's also a contention.

    reply to this | link to this | view in chronology ]

  • identicon
    davidlow, 24 May 2006 @ 8:50am

    What ever happened to Zoemail??

    I'm surprised I never see any mention of Zoemail.com as a
    complete and absolute solution to spam problems. It's been doing
    the job for me for over two years. There is no filtering, so there's
    no danger of blocking any friendlies. None. I thought for sure it
    would catch on. Am I alone?

    I just posted the same comment over at CNet's version of the same story.

    reply to this | link to this | view in chronology ]

    • identicon
      mthorn, 24 May 2006 @ 10:24am

      Re: What ever happened to Zoemail??

      Once spammers catch on to this service they will simply send to "Sally@example.com" and "Sally.realword@example.com". What will you do then?

      reply to this | link to this | view in chronology ]

    • identicon
      mthorn, 24 May 2006 @ 10:29am

      Re: What ever happened to Zoemail??

      The spam filter I worked on had a similar service. It allowed you to make throwaway emails, a random string in front of your domain. Once the email was used for a limited number of times, or a limited amount of time (your preference), it was deleted. You would use these trash emails at retailers and such that are guaranteed to spam you, but you need to get a confirmation email.

      reply to this | link to this | view in chronology ]

  • identicon
    joe stacey, 24 May 2006 @ 9:08am

    The vendor should have been aware of the unreliability of email. If they don't know this, they shouldn't have a business. It's basic stuff.

    Anyone business owner with a brain should know to follow up on an email of this level of importance.

    derhh...

    reply to this | link to this | view in chronology ]

  • identicon
    mthorn, 24 May 2006 @ 10:27am

    Whitelist words

    Good spam filters support whitelist words. If the word is in the subject or body of the email, the mail proceeds, NEVER being filtered. The word could be your business name, and address, a phone number, or a nonsense word. You would put this keyword in your outbound message so that the reply would have it and go through.

    Blocking real spam is easy, spammers only use a few tricks to get around spam filters, these tricks are so obvious that they give it away as being spam. Of course you need a qualified human at the filtering end to pick up on new trends and key these in. This is where spam filters fail, no one wants to pay someone to maintain the spam filter.

    reply to this | link to this | view in chronology ]

  • identicon
    Another District, 24 May 2006 @ 12:30pm

    I Feel Their Pain

    I am the IT Director of a very large school district. Most people posting comments here have obviously never managed a very large e-mail system.

    We use a spam filter for our district's e-mail system. The filter rules, white lists, black lists, etc. are downloaded daily from the manufacturer of the filter (very much like antivirus software). We estimate the filter to be about 90% effective removing junk e-mail. We block approximately 60,000 messages per day. It is not the same thing as desktop junk e-mail software that you all are saying takes a few minutes to check each day. The mail is filtered before it reaches the mailbox. There's simply no practical way to check it for "good" e-mail every day. And for those of you that are not aware, most school districts have an IT staff level about 80% smaller than corporations of similar size. So before you all condemn the school district for missing the e-mail, you should realize we aren't talking about desktop filters and staff members that have time to review 60,000 messages per day.

    Filtering systems are a fact of life, and like web filters, junk mail filters are not perfect. The more effective they are at blocking junk e-mail, the more likely they are to accidentally block good e-mail. Security comes at a cost of convenience and usability. And this is one of the disadvantages of spam filters.

    reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 24 May 2006 @ 1:36pm

      Re: I Feel Their Pain

      severely understaffed, but as head of the department you seem to have time to surf the net for worthwhile tripe like this.

      And yet the schools demand milliions more dollars because they are sooooo overworked.

      Pathetic.

      reply to this | link to this | view in chronology ]

      • identicon
        Anonymous Howard, 24 May 2006 @ 3:17pm

        Re: Re: I Feel Their Pain

        Maybe before judging him and assuming the worst you might look at the time stamp. 12:30 PM. MAYBE he was on lunch break. Maybe not, I don't know what time zone he posted from. Maybe he uses Techdirt to keep on top of the situation so his school distrcit is aware of these problems. Maybe (more likely Probably) you're just an arse!

        reply to this | link to this | view in chronology ]

    • identicon
      mthorn, 24 May 2006 @ 1:37pm

      Re: I Feel Their Pain

      Each recipient doesn't get 60,000 messages a day. The recipient should be responsible for determining if a message is good or not. This drastically reduces the amount of messages that need to be checked per user. It's not the IT departments job to determine the validity of an email, they don't know what the message is about, nor should they.

      Spam filters can also determine if it is most likely spam, or definitly spam. The definites (the majority of spam) do not need to be searched by a user unless they can't find the message they are expecting in their inbox.

      Reguardless of the amount of spam or how it is determined to be spam, the recipient should have a way to find it, most conveintly on a web interface. It should not be up to the IT department to trash messages they think is spam.

      I worked on a system that had hundreds of thousands of junk email a day, each one archived. This archived came in handy many times when messages were inappropriatly blocked. This became and invaluable tool for use to determine why it was falsely blocked and how to prevent it again.

      If your answer is "we have so much spam our only choice is to delete it", then you are using the wrong software, or you don't have the staff qualified to answer the question correctly.

      reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 24 May 2006 @ 1:40pm

      Re: I Feel Their Pain

      Did you read that description of the spam filter off the back of the box you got at Best Buy?

      Maybe if you employed an internet filter as well, teachers and administrators wouldn't be surfing the net all day eliciting spam from retailers.

      reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 25 May 2006 @ 10:36am

    What if someone wanted to rig the bidding process? Requiring responses by email and then using filters to selectively delete certain replies would be an effective way of doing so and then denying culpability.

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 8 Jun 2006 @ 5:31am

    Yet another example of the judicial system gone awry.

    The public and the law is so highly against SPAM and malicious activity that if we do not implement protection we could be held very accountable if who knows what could happen, especially in an educational institution, yet, when these same tools block a single email that institution is fined for doing what it has to do.

    No one can be held accountable for using SPAM filters. Period. End of story. If you are relying so heavily on e-mail then you have absolutely no grasp of reality and definately don't deserve your job.

    reply to this | link to this | view in chronology ]


Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here



Subscribe to the Techdirt Daily newsletter




Comment Options:

  • Use markdown. Use plain text.
  • Remember name/email/url (set a cookie)

Close

Add A Reply

Have a Techdirt Account? Sign in now. Want one? Register here



Subscribe to the Techdirt Daily newsletter




Comment Options:

  • Use markdown. Use plain text.
  • Remember name/email/url (set a cookie)

Follow Techdirt
Techdirt Gear
Shop Now: Copying Is Not Theft
Advertisement
Report this ad  |  Hide Techdirt ads
Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Chat
Advertisement
Report this ad  |  Hide Techdirt ads
Recent Stories
Advertisement
Report this ad  |  Hide Techdirt ads

Close

Email This

This feature is only available to registered users. Register or sign in to use it.