“Aaron’s Censorware” demonstrates that the core technology behind a filtering proxy is pretty straightforward. After all, a lot of what’s used for anti-spam can be used in content filtering, just applying the same techniques (e.g., Bayesian analysis) to a different data stream.

This will soon devolve into a battle over what constitutes adequate protection measures. Is a blacklist sufficient, and if so, who determines who’s on the blacklist? Do we need to scan for certain terms, and if so, who determines what terms are considered inappropriate?

After all, if the Pennsylvania Attorney General can order ISPs to block certain Web sites from being viewed by Pennsylvania residents, it’s not that far of a leap to infer that governments will feel they have the ability to control the nature of the filtering, not just that filtering exists.

I’m not a librarian (nor do I play one on TV), but it would seem what’s needed is a setup akin to spam filtering:

— Open source filtering proxy that is easy for librarians to configure

— Ability to “subscribe” to any number of site or term blacklists, created by firms or volunteer organizations, much like the spam blacklists that exist today, with simple lists available “out of the box” for simple installation (just like one subscribes to anti-virus signature updates)

— Ability to have local overrides to the subscribed-to lists, to either specifically allow or specifically deny certain sites or terms, to help tailor for local issues

— Configurable record-keeping (log nothing, log with X days retention, log everything)

The key is the subcribe-able blacklists. That’s what the various commercial filtering firms really provide. By having open standards on how one subscribes to and receives blacklist updates, it becomes fairly easy for different groups to decide to provide their own take on the blacklists. Libraries (or anyone else using this) could “dial in” their filtering based on blacklist provider reputation (e.g., blacklists provided by a tradtional religious group might differ from a blacklist from the ACLU).

My question is: does anyone really have interest in this? As a technologist, I can envision really cool solutions, but I have no sense of whether this problem really exists.

(BTW, this is being cross-posted to my personal blog)