US Government Admits: Data Mining For Terrorists Doesn't Work

from the oops dept

Over the last few years, you probably have noticed multiple attempts by state and federal government agencies to collect more data on people and to share that data more widely. This is troubling for a variety of reasons, often having to do with privacy and questions concerning the quality of data, but those concerns are almost always pushed aside by claims that this data is necessary for fighting terrorism. There’s this myth out there that if we just had more data, somehow we’d be able to stop terrorists. The problem is that this is untrue. More data often makes it even harder to find the important data, and now a detailed government report has basically confirmed that data mining doesn’t help in finding terrorists.

The report was put together by a bunch of well-respected academics, industry insiders and law enforcement officials on behalf of the National Research Council, and finds that the idea of plugging all this data into a machine and popping out terrorist identities is a myth. The report then lists out a variety of recommendations, concerning the use of any data mining systems, and how to make sure that individual privacy is protected. One key finding that will never be implemented: when using any such system, systematic reviews should be done to see if the systems are actually effective. If there’s one thing we’ve seen with government programs over the years, such attempts to actually review something they’ve put in place to see if it actually worked are almost never done.

Filed Under: , , ,

Rate this comment as insightful
Rate this comment as funny
You have rated this comment as insightful
You have rated this comment as funny
Flag this comment as abusive/trolling/spam
You have flagged this comment
The first word has already been claimed
The last word has already been claimed
Insightful Lightbulb icon Funny Laughing icon Abusive/trolling/spam Flag icon Insightful badge Lightbulb icon Funny badge Laughing icon Comments icon

Comments on “US Government Admits: Data Mining For Terrorists Doesn't Work”

Subscribe: RSS Leave a comment
Anonymous Coward says:

Re: Re: Re:

Hmm. That seems to be a problem- is that no computer can replace that which a human-to-human interaction has. Frankly, I don’t care. I may have alternative political views to yours, but does that make me an enemy? Depends on who is in charge, I guess.

Those that have met me know I have the best intentions, but what real emotion can be created and shared through a computer screen? It’s always subject to the receptiveness to recieving side. If someone wants to defame me based on data mined political or social views, fine. Want to see who I donate to? Go for it. Chances are, you are probably close enough to me to just pick up the fucking phone and ask.

You, sir, are the coward. So have fun watching me as I have fun exploring the world and interwebs. Maybe you’ll learn something too.

Mr Big Content says:

Reality Has A Liberal Bias

I don’t buy all this “research” and “evidence” baloney. Look for the “truth”, and you’re always going to come up with a “true” conclusion. And what is “truth”, but just another kind of namby-pamby pinko liberal bias?

No, the way to come up with the right, God-fearing answer, is to leave no stone unturned in your determination to turn up that answer. If these researchers had the balls to do that, they would have seen the light, that what the Government is doing is the only sane way to make sure the TERRRISTS don’t take over this country.

oneofmany says:

no way!

really? what a surprise! maybe it has to do with the fact that unlike those same academics and government overpaid employees the terrorist don’t really use a computer all the time post on blogs and such. You can’t data mine when you don’t have digital data. Plus the effectiveness of data mining in many cases is quite controversial.

Anonymous Coward says:

Failure will continue...

…because you can’t model human behavior with math.

They’ve tried this a few times- Enron was the first iteration (learning about what’s necessary for business buy-in) The current financial collapse was the second iteration (where personal information was used to determine and mitigate credit risk)

If this doesn’t stop, and return to fundamentals… think about it.

Who are “The smartest guys in the room”?

Isaac K (profile) says:

Uhmn, as a government worker...

If there’s one thing we’ve seen with government programs over the years, such attempts to actually review something they’ve put in place to see if it actually worked are almost never done.

I guess I shouldn’t be surprised the general public doesn’t know this, but the OIG and GAO are institutions created to specifically perform such reviews on a constant basis.

And I assure you, they are VERY thorough. In my agency, the operating budget is less than 2% of our total outlays, basically unheard of anywhere, government or private, except for some non-profit orgs. Efficiency? Large government may not be the MOST efficient method, but a large part of the cost is the intervening layers filtering from a national system down to the local level. And that is where most of the problems, trickery, etc. come into play.

All-in-all, I agree though; it would be very hard to determine the effectiveness of such a data filtration system.
If test returns a negative result, it might just be that the data you are looking for doesn’t exist, rather than a problem with the aggregator/filter.

NullOp says:

Under optimal conditions is almost impossible...

If you have ever done any sort of survaillance you will know how much “junk” you come up with. Now, if you monitor “everything” you will realize how faint the sound of terrorism is in the static of everyday life/communications. In short, its a lot like listening for ET.

BTW, yes, you can model human behavior with math. Just not as well as Wal-Mart would like to be able to do.

Pelroston says:

Other Purposes

…data-mining is quite useful to the government for purposes ‘other’ than finding terrorists.

Heavy surveillance of the domestic population (.. especially political opponents & non-cooperative citizens) is greatly aided by data-mining.

Extending political control is always the objective of all government actors.

Assuming that anti-terrorism is the only objective of data-mining… is a HUGE mistake.

Add Your Comment

Your email address will not be published. Required fields are marked *

Have a Techdirt Account? Sign in now. Want one? Register here

Comment Options:

Make this the or (get credits or sign in to see balance) what's this?

What's this?

Techdirt community members with Techdirt Credits can spotlight a comment as either the "First Word" or "Last Word" on a particular comment thread. Credits can be purchased at the Techdirt Insider Shop »

Follow Techdirt

Techdirt Daily Newsletter

Techdirt Deals
Techdirt Insider Discord
The latest chatter on the Techdirt Insider Discord channel...