<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0"
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
xmlns:wfw="http://wellformedweb.org/CommentAPI/">
<channel>
<title>Techdirt. Stories filed under &quot;algorithms&quot;</title>
<description>Easily digestible tech news...</description>
<link>http://www.techdirt.com/</link>
<language>en-us</language>
<image><title>Techdirt. Stories filed under &quot;algorithms&quot;</title><url>http://www.techdirt.com/images/td-88x31.gif</url><link>http://www.techdirt.com/</link></image>
<item>
<pubDate>Thu, 9 May 2013 17:00:00 PDT</pubDate>
<title>DailyDirt: Can Computers Grade Written Essays?</title>
<dc:creator>Michael Ho</dc:creator>
<link>http://www.techdirt.com/articles/20110226/12421713271/dailydirt-can-computers-grade-written-essays.shtml</link>
<guid>http://www.techdirt.com/articles/20110226/12421713271/dailydirt-can-computers-grade-written-essays.shtml</guid>
<description><![CDATA[ Technology aimed at education could really benefit an incredible number of students by making classes and learning (potentially) a more pleasant and efficient experience. Computers can't replace a really good human teacher, but they can make it easier for good human teachers to reach a vast audience of students. Massively open online courses (MOOCs) promise to change how education works, but there are some technological tools that might be missing. It's pretty straightforward to test students on math problems in an automated way, but grading essays is a much more daunting problem. There have been some <a href="http://www.hewlett.org/newsroom/press-release/hewlett-foundation-sponsors-prize-improve-automated-scoring-student-essays">calls for automated grading software</a> from various organizations (like the Hewlett Foundation). 
But at the same time, the National Council of Teachers of English argues that <a href="http://www.washingtonpost.com/blogs/answer-sheet/wp/2013/04/25/can-computers-really-grade-essay-tests/">computers simply can't grade essays</a>. Here are just a few more links on this debate over the use of algorithms over English professors (or grad students).

<ul>

<li> <a title="http://www.nytimes.com/2013/04/05/science/new-test-for-computers-grading-essays-at-college-level.html" href="http://nyti.ms/18PxUci">EdX, the non-profit started by Harvard and MIT, is releasing some software to automagically grade human-written essays.</a> Some see this software as just another tool for educators to use for more immediate feedback to students, while others are <a href="http://humanreaders.org/petition/">worried</a> that these algorithms will be used incorrectly and lead to disastrous educational policies and outcomes. [<a href="http://www.nytimes.com/2013/04/05/science/new-test-for-computers-grading-essays-at-college-level.html">url</a>]</li>

<li> <a title="http://mfeldstein.com/si-ways-the-edx-announcement-gets-automated-essay-grading-wrong/" href="http://bit.ly/18PylmX">There are studies that show algorithms are statistically comparable to humans when it comes to ranking essays on a 5 point scale.</a> There are things machines can do better and things humans do better -- just make sure you know the differences and automated essay grading can be done productively in the right context. [<a href="http://mfeldstein.com/si-ways-the-edx-announcement-gets-automated-essay-grading-wrong/">url</a>]</li>

<li> <a title="http://www.nytimes.com/2012/04/23/education/robo-readers-used-to-grade-test-essays.html?pagewanted=all" href="http://nyti.ms/YBjXQf">Automated essay readers can grade 16,000 essays in 20 seconds.</a> The Educational Testing Service is testing out automation, so students may soon be facing algorithmic grading for their college entrance exams. [<a href="http://www.nytimes.com/2012/04/23/education/robo-readers-used-to-grade-test-essays.html?pagewanted=all">url</a>]</li>

<li> <a title="http://www.washingtonpost.com/blogs/answer-sheet/wp/2013/05/02/grading-writing-the-art-and-science-and-why-computers-cant-do-it/" href="http://wapo.st/10f06Ax">Grading a few sentences can be harder than it might look.</a> Professional (human) teachers are obviously better at interpreting the insights and ideas behind the words a student writes, but computers scale much better and never tire of horrible spelling mistakes or misplaced modifiers.... [<a href="http://www.washingtonpost.com/blogs/answer-sheet/wp/2013/05/02/grading-writing-the-art-and-science-and-why-computers-cant-do-it/">url</a>]</li>

</ul>

If you'd like to read more awesome and interesting stuff, check out this unrelated (but not entirely random!) <a title="http://www.stumbleupon.com/to/stumble/stumblethru:www.techdirt.com" href="http://bit.ly/fagV8c">Techdirt post</a> via StumbleUpon.<br /><br /><a href="http://www.techdirt.com/articles/20110226/12421713271/dailydirt-can-computers-grade-written-essays.shtml">Permalink</a> | <a href="http://www.techdirt.com/articles/20110226/12421713271/dailydirt-can-computers-grade-written-essays.shtml#comments">Comments</a> | <a href="http://www.techdirt.com/articles/20110226/12421713271/dailydirt-can-computers-grade-written-essays.shtml?op=sharethis">Email This Story</a><br />
 ]]></description>
<slash:department>urls-we-dig-up</slash:department>
<wfw:commentRss>http://www.techdirt.com/comment_rss.php?sid=20110226/12421713271</wfw:commentRss>
</item>
<item>
<pubDate>Wed, 3 Apr 2013 00:03:13 PDT</pubDate>
<title>Programming The News: The Future Of Reporting Is Algorithms</title>
<dc:creator>Tim Cushing</dc:creator>
<link>http://www.techdirt.com/articles/20130331/21015322519/programming-news-future-reporting-is-algorithms.shtml</link>
<guid>http://www.techdirt.com/articles/20130331/21015322519/programming-news-future-reporting-is-algorithms.shtml</guid>
<description><![CDATA[ <p>
This may seem like the sort of statement usually delivered by an overblown narrator as rockets and lasers go zooming* by, but here goes: In the world of journalism, the future is now! Granted, it's the kind of future that often makes waves in the present and raises at least as many questions as it answers, but if you wanted a bright, problem-free future, you'd have to travel back to the divergence point somewhere between Philip K. Dick and The Jetsons... and then eliminate the dystopians.
<br /><br />
<i>*Yes, I realize lasers don't make noise or "zoom" by, but that hasn't prevented George Lucas from becoming insanely rich, has it?</i>
<br /><br />
But you can't, <a href="http://www.vancouversun.com/news/story.html?id=8156059" target="_blank">so here we are, discussing journalism... by robots!</a> [INS FANFARE/LASER NOISES]
<blockquote>
<i>Journalist Ken Schwencke has occasionally awakened in the morning to find his byline atop a news story he didn&rsquo;t write.</i>
<br /><br />
<i>No, it&rsquo;s not that his employer, The Los Angeles Times, is accidentally putting his name atop other writers&rsquo; articles. Instead, it&rsquo;s a reflection that Schwencke, digital editor at the respected U.S. newspaper, wrote an algorithm &mdash; that then wrote the story for him.</i>
<br /><br />
<i>Instead of personally composing the pieces, Schwencke developed a set of step-by-step instructions that can take a stream of data &mdash; this particular algorithm works with earthquake statistics, since he lives in California &mdash; compile the data into a pre-determined structure, then format it for publication.</i>
<br /><br />
<i>His fingers never have to touch a keyboard; he doesn&rsquo;t have to look at a computer screen. He can be sleeping soundly when the story writes itself.</i></blockquote>
This isn't exactly <i>new</i> news. (Then again, neither is the morning paper, but that's a discussion for another time...) Algorithmic story generation has been around for a few years now, with Narrative Science leading the field. A couple of years ago, Narrative Science <i>was</i> the story, rather than just the automated recap. George Washington University's website had covered a GWU baseball game with a longish recap that only got around to mentioning the <i>opposing pitcher's perfect game</i> in the <i>seventh</i> (out of eight) paragraph. Speculators wondered if a bot was behind this "ignoring the forest for the trees" recap. Narrative Science's techies were highly offended and responded by producing <a href="http://www.techdirt.com/articles/20110418/01013313932/robot-journalist-writes-better-story-college-baseball-perfect-game.shtml" target="_blank">two algorithmically-generated recaps</a> -- one from the home team POV and a more neutral piece.
<br /><br />
The first concern with robo-journalism is often expressed by the journalists themselves: are we getting pushed out?
<br /><br />
<a href="http://www.wired.com/gadgetlab/2012/04/can-an-algorithm-write-a-better-news-story-than-a-human-reporter/all/" target="_blank">Kristian Hammond, co-founder of Narrative Science, doesn't see it that way</a>.
<blockquote>
<i>This robonews tsunami, he insists, will not wash away the remaining human reporters who still collect paychecks. Instead the universe of newswriting will expand dramatically, as computers mine vast troves of data to produce ultracheap, totally readable accounts of events, trends, and developments that no journalist is currently covering.</i></blockquote>
This is somewhat echoed by L.A. Times reporter Schwencke, who sees the algorithmic output as a boon for busy journalists.
<blockquote>
<i>Schwencke says the use of algorithms on routine news tasks frees up professional reporters to make phone calls, do actual interviews, or dig through sophisticated reports and complex data, instead of compiling basic information such as dates, times and locations.</i>
<br /><br />
<i>&ldquo;It lightens the load for everybody involved,&rdquo; he said.</i></blockquote>
Schwenke's "bot" is rather simple, functioning best with a limited dataset and a minimum of formatting. Narrative Science's output is a bit more complex, allowing customers to adjust the "slant" of the generated stories. Not only that, but the software can cop an attitude, if requested.
<blockquote>
<i>The Narrative Science team also lets clients customize the tone of the stories. &ldquo;You can get anything, from something that sounds like a breathless financial reporter screaming from a trading floor to a dry sell-side researcher pedantically walking you through it,&rdquo; says Jonathan Morris, COO of a financial analysis firm called Data Explorers, which set up a securities newswire using Narrative Science technology. (Morris ordered up the tone of a well-educated, straightforward financial newswire journalist.) Other clients favor bloggy snarkiness. &ldquo;It&rsquo;s no more difficult to write an irreverent story than it is to write a straightforward, AP-style story,&rdquo; says Larry Adams, Narrative Science&rsquo;s VP of product. &ldquo;We could cover the stock market in the style of Mike Royko.&rdquo;</i></blockquote>
This leads to the ethical quandary presented by the use of bots. Is robo-generated journalism really journalism, and is the use of algorithms a betrayal of readers' trust, especially when a familiar name is on the byline? If factual errors are discovered, does the blame lie with the software, or with the journalist who agreed to let the article "write itself?"
<br /><br />
The answer here isn't simple (and the question likely isn't even fully formed yet), but the key is transparency.
<blockquote>
<i>&ldquo;People are already reading automated data reports that come to them, and they don&rsquo;t think anything of it,&rdquo; said Ben Welsh, a colleague of Schwencke&rsquo;s at the Times.</i>
<br /><br />
<i>Welsh says that responsibility for accuracy falls where it always has: with publications, and with individual journalists.</i>
<br /><br />
<i>&ldquo;The key thing is just to be honest and transparent with your readers, like always,&rdquo; he said. &ldquo;I think that whether you write the code that writes the news or you write it yourself, the rules are still the same.&rdquo;</i>
<br /><br />
<i>&ldquo;You need to respect your reader. You need to be transparent with them, you need to be as truthful as you can&hellip; all the fundamentals of journalism just remain the same.&rdquo;</i></blockquote>
Questions involving intellectual property are also raised, although they aren't discussed in these articles. Who holds the copyright on the generated articles? In Schwencke's case, these rights are likely retained by the L.A. Times. In the case of Narrative Science, it's probably defined by contractual terms with the end user. Once the contract is up, the generated articles' copyright reverts to the end user.
<br /><br />
Schwencke's homebrewed algorithm is a different IP animal. If he switches papers, does he retain the right to the "bot?" Or is that algorithm, developed while employed with the L.A. Times, considered a "work for hire," and thus, the paper's property? Arguably, his algorithm is an extension of <i>him</i>, covering his area of expertise and designed to emulate <i>his</i> reporting. What if Schwencke generates a similar piece of software for his new employer? Would he be permitted to do this, or would this be prevented by additions to "non-compete" clauses? Is it patentable?
<br /><br />
The more ubiquitous "robo-journalism" becomes, the more issues like these will arise. Hopefully, IP turf wars will remain at a minimum, allowing for the expansion of this promising addition to the journalist's toolset. With bots handling basic reporting, journalists should be freed up to pursue the sort of journalism you can't expect an algorithm to handle -- longform, investigative, etc. This is good news for readers, even if they may find themselves a little unnerved (at first) by the journalistic uncanny valley.
<br /><br />
</p><br /><br /><a href="http://www.techdirt.com/articles/20130331/21015322519/programming-news-future-reporting-is-algorithms.shtml">Permalink</a> | <a href="http://www.techdirt.com/articles/20130331/21015322519/programming-news-future-reporting-is-algorithms.shtml#comments">Comments</a> | <a href="http://www.techdirt.com/articles/20130331/21015322519/programming-news-future-reporting-is-algorithms.shtml?op=sharethis">Email This Story</a><br />
 ]]></description>
<slash:department>I-for-one-welcome-our-new-fedora-clad-robotic-overlords</slash:department>
<wfw:commentRss>http://www.techdirt.com/comment_rss.php?sid=20130331/21015322519</wfw:commentRss>
</item>
<item>
<pubDate>Wed, 9 Jan 2013 17:00:00 PST</pubDate>
<title>DailyDirt: Adapting To A TL;DR Future</title>
<dc:creator>Michael Ho</dc:creator>
<link>http://www.techdirt.com/articles/20101111/18110611828/dailydirt-adapting-to-tldr-future.shtml</link>
<guid>http://www.techdirt.com/articles/20101111/18110611828/dailydirt-adapting-to-tldr-future.shtml</guid>
<description><![CDATA[ With the ever growing amount of content to read/watch/hear, humans are going to need to figure out much more efficient ways to consume information. Speed reading will only get us so far, and then we're going to have to rely on ways to filter out irrelevant stuff so we can focus more on just the things that we really need to see. Here are just a few early attempts to get computers to help us out with information overload.

<ul>

<li> <a title="http://www.fastcompany.com/3004507/15-year-old-built-app-help-his-high-school-debate-team-it-could-do-much-more" href="http://bit.ly/XkMe8G">Clipped is a software tool that tries to extract key bits of information from long pieces of text.</a> Tanay Tandon, a 15yo kid, created it and also filed a patent for his algorithm.... [<a href="http://www.fastcompany.com/3004507/15-year-old-built-app-help-his-high-school-debate-team-it-could-do-much-more">url</a>]</li>

<li> <a title="http://reviews.cnet.com/software/tldr/4505-3513_7-35567338.html" href="http://cnet.co/VPPDuE">TLDR is a browser plug-in that tries to create short summaries for articles.</a> These algorithms aren't perfect, but then again, neither is human comprehension. [<a href="http://reviews.cnet.com/software/tldr/4505-3513_7-35567338.html">url</a>]</li>

<li> <a title="http://smallbusiness.yahoo.com/advisor/world-s-youngest-vc-funded-entrepreneur-.html" href="http://yhoo.it/WtT6xV">Another teenager wrote news-summarizing software and raised $1 million for it.</a> It was originally called Trimmit, but now it's Summly -- and 16yo Nick D'Alosio is looking for some "serious scientists" to help him improve his algorithms with that VC funding. [<a href="http://smallbusiness.yahoo.com/advisor/world-s-youngest-vc-funded-entrepreneur-.html">url</a>]</li>

</ul>

If you'd like to read more awesome and interesting stuff, check out this unrelated (but not entirely random!) <a title="http://www.stumbleupon.com/to/stumble/stumblethru:www.techdirt.com" href="http://bit.ly/fagV8c">Techdirt post</a>.<br /><br /><a href="http://www.techdirt.com/articles/20101111/18110611828/dailydirt-adapting-to-tldr-future.shtml">Permalink</a> | <a href="http://www.techdirt.com/articles/20101111/18110611828/dailydirt-adapting-to-tldr-future.shtml#comments">Comments</a> | <a href="http://www.techdirt.com/articles/20101111/18110611828/dailydirt-adapting-to-tldr-future.shtml?op=sharethis">Email This Story</a><br />
 ]]></description>
<slash:department>urls-we-dig-up</slash:department>
<wfw:commentRss>http://www.techdirt.com/comment_rss.php?sid=20101111/18110611828</wfw:commentRss>
</item>
<item>
<pubDate>Tue, 16 Oct 2012 17:00:00 PDT</pubDate>
<title>DailyDirt: Looking For Love In Some Of The Wrong Places</title>
<dc:creator>Michael Ho</dc:creator>
<link>http://www.techdirt.com/articles/20100711/21351810161/dailydirt-looking-love-some-wrong-places.shtml</link>
<guid>http://www.techdirt.com/articles/20100711/21351810161/dailydirt-looking-love-some-wrong-places.shtml</guid>
<description><![CDATA[ Online dating is by no means a new thing anymore, and by some <a href="http://www.northwestern.edu/newscenter/stories/2012/02/online-dating-finkel.html">counts</a>, we're on the third iteration of improvement for internet dating. So that means we should be pretty close to perfecting these services, right? (Third time's the charm?) Matching algorithms will probably get better and better with time, but then so will expectations. Here are just a few interesting links for geeky singles out there.

<ul>

<li> <a title="http://www.scientificamerican.com/article.cfm?id=scientific-flaws-online-dating-sites" href="http://bit.ly/RCL87O">Psychologists have pointed out that matching algorithms for long-term relationships are not significantly better than random.</a> Dating algorithms aren't actually so bad at excluding potentially "bad dates" -- but that's not what most online dating services offer. [<a href="http://www.scientificamerican.com/article.cfm?id=scientific-flaws-online-dating-sites">url</a>]</li>

<li> <a title="http://bigthink.com/dollars-and-sex/no-winners-in-an-online-dating-tournament" href="http://bit.ly/RCKUgR">The Secret Diamond Club takes advantage of some economic tricks to try to match up rich men with attractive women.</a> The real secret, though, is that it probably doesn't work at all -- and it preys on lonely people with money. [<a href="http://bigthink.com/dollars-and-sex/no-winners-in-an-online-dating-tournament">url</a>]</li>

<li> <a title="http://www.cbsnews.com/8301-505123_162-57532993/a-nobel-for-work-that-affects-your-daily-life/" href="http://cbsn.ws/RzdaO7">Recently, the Nobel prize for economics was awarded for work on matching markets.</a> The Gale-Shapely algorithm has been used for matching organ donors and doctors with hospitals, but maybe someday it'll be used for finding romantic partners, too. [<a href="http://www.cbsnews.com/8301-505123_162-57532993/a-nobel-for-work-that-affects-your-daily-life/">url</a>]</li>

</ul>

If you'd like to read more awesome and interesting stuff, check out this unrelated (but not entirely random!) <a title="http://www.stumbleupon.com/to/stumble/stumblethru:www.techdirt.com" href="http://bit.ly/fagV8c">Techdirt post</a>.<br /><br /><a href="http://www.techdirt.com/articles/20100711/21351810161/dailydirt-looking-love-some-wrong-places.shtml">Permalink</a> | <a href="http://www.techdirt.com/articles/20100711/21351810161/dailydirt-looking-love-some-wrong-places.shtml#comments">Comments</a> | <a href="http://www.techdirt.com/articles/20100711/21351810161/dailydirt-looking-love-some-wrong-places.shtml?op=sharethis">Email This Story</a><br />
 ]]></description>
<slash:department>urls-we-dig-up</slash:department>
<wfw:commentRss>http://www.techdirt.com/comment_rss.php?sid=20100711/21351810161</wfw:commentRss>
</item>
<item>
<pubDate>Thu, 11 Oct 2012 17:00:00 PDT</pubDate>
<title>DailyDirt: Games Played By Computers</title>
<dc:creator>Michael Ho</dc:creator>
<link>http://www.techdirt.com/articles/20101011/04435411362/dailydirt-games-played-computers.shtml</link>
<guid>http://www.techdirt.com/articles/20101011/04435411362/dailydirt-games-played-computers.shtml</guid>
<description><![CDATA[ Artificial intelligence projects have come a long way with algorithms that can beat some of the best humans at poker and chess and a whole bunch of other games. There are still a few games that people can clearly play better than computers, but that list is getting shorter all the time. Here are just a few bots that are learning how to beat us at our own games.

<ul>
<li> <a title="http://www.newscientist.com/article/mg21528726.300-computer-watches-you-play-a-game-then-beats-you-at-it.html" href="http://bit.ly/Vu1F0b">French researchers are working on software that can learn how to play a game just by watching humans play -- and their software has mastered games like tic-tac-toe and connect4.</a> More complex games like chess are a bit too difficult, but maybe someday learning algorithms will be able to take on arbitrary games. [<a href="http://www.newscientist.com/article/mg21528726.300-computer-watches-you-play-a-game-then-beats-you-at-it.html">url</a>]</li>

<li> <a title="http://www.telegraph.co.uk/technology/technology-video/9358901/Scissors-Paper-Stone-unbeatable-robot-wins-every-hand.html" href="http://bit.ly/ST4AJc">Playing Rock-Paper-Scissors with a robot hand doesn't sound like a particularly winnable game -- especially when the robot cheats all the time.</a> This robot only takes about a millisecond to detect what a human hand will play, so the only winning strategy for humans is to try to come up with deceptive ways to play RPS. [<a href="http://www.telegraph.co.uk/technology/technology-video/9358901/Scissors-Paper-Stone-unbeatable-robot-wins-every-hand.html">url</a>]</li>

<li> <a title="http://www.utexas.edu/news/2012/09/26/artificially-intelligent-game-bots-pass-the-turing-test-on-turing%E2%80%99s-centenary/" href="http://bit.ly/TiaQe0">A panel of judges couldn't tell the difference between a computer player and a human player in an Unreal tournament (in a strange kind of Turing test).</a> Non-player characters controlled by algorithms are getting more and more human-like, and pretty soon humans won't even need to play video games anymore.... [<a href="http://www.utexas.edu/news/2012/09/26/artificially-intelligent-game-bots-pass-the-turing-test-on-turing%E2%80%99s-centenary/">url</a>]</li>

</ul>

If you'd like to read more awesome and interesting stuff, check out this unrelated (but not entirely random!) <a title="http://www.stumbleupon.com/to/stumble/stumblethru:www.techdirt.com" href="http://bit.ly/fagV8c">Techdirt post</a>.<br /><br /><a href="http://www.techdirt.com/articles/20101011/04435411362/dailydirt-games-played-computers.shtml">Permalink</a> | <a href="http://www.techdirt.com/articles/20101011/04435411362/dailydirt-games-played-computers.shtml#comments">Comments</a> | <a href="http://www.techdirt.com/articles/20101011/04435411362/dailydirt-games-played-computers.shtml?op=sharethis">Email This Story</a><br />
 ]]></description>
<slash:department>urls-we-dig-up</slash:department>
<wfw:commentRss>http://www.techdirt.com/comment_rss.php?sid=20101011/04435411362</wfw:commentRss>
</item>
<item>
<pubDate>Fri, 22 Jun 2012 09:12:00 PDT</pubDate>
<title>Speech-Via-Algorithm Is Still Speech, And Censoring It Is Still Censorship</title>
<dc:creator>Mike Masnick</dc:creator>
<link>http://www.techdirt.com/articles/20120621/14440519421/speech-via-algorithm-is-still-speech-censoring-it-is-still-censorship.shtml</link>
<guid>http://www.techdirt.com/articles/20120621/14440519421/speech-via-algorithm-is-still-speech-censoring-it-is-still-censorship.shtml</guid>
<description><![CDATA[ I have to admit that I'm really confused about law professor and FTC advisor Tim Wu's general position on a variety of topics lately.  In the <a href="http://www.techdirt.com/articles/20040802/0149210.shtml">past</a>, I've often found him to be thoughtful and (very) insightful on various topics concerning internet policy and regulations.  But, at times, he seems to go off the deep end, such as with his recent claims that <a href="http://www.techdirt.com/articles/20101116/01370411876/tim-wu-insists-that-market-domination-is-a-monopoly.shtml">big</a> automatically means a monopoly.  But, his latest piece in the NYTimes goes way further than anything I've seen before: claiming that <a href="http://www.nytimes.com/2012/06/20/opinion/free-speech-for-computers.html?_r=1" target="_blank">search results shouldn't get First Amendment protection</a> because it's "computers" speaking, not humans.
<blockquote><i>
Is there a compelling argument that computerized decisions should be considered speech? As a matter of legal logic, there is some similarity among Google, Ann Landers, Socrates and other providers of answers. But if you look more closely, the comparison falters. Socrates was a man who died for his views; computer programs are utilitarian instruments meant to serve us. Protecting a computer&#8217;s &#8220;speech&#8221; is only indirectly related to the purposes of the First Amendment, which is intended to protect actual humans against the evil of state censorship. The First Amendment has wandered far from its purposes when it is recruited to protect commercial automatons from regulatory scrutiny. 
</i></blockquote>
This is wrong.  And dangerous.
<br /><br />
Let's be clear here: what search engines do is present opinions.  They are opinions based on data programmed <i>by humans.</i>  There is nothing special in that it's a computer that cranks through the data to output the opinion.  Taken to its logical conclusion, Wu's argument is that we should protect uninformed opinions not based on an algorithmic exploration of the data -- but the second you add in a computer to crunch the numbers, that opinion is no longer protected.  Contrary to Wu's assertion that this "wanders" from the point of the First Amendment, I'd argue the exact opposite.  The First Amendment should protect all kinds of speech, but we should be especially happy about that which is based on data.
<br /><br />
Two responses to Wu's piece help highlight this point nicely.  Julian Sanchez notes that Wu's argument seems to suggest that <a href="http://www.cato-at-liberty.org/were-not-censoring-you-just-your-computer/" target="_blank">any computer generated content</a> doesn't get First Amendment protections -- and that would include computer-generated video games and movies:
<blockquote><i>
Consider an argument for denying First Amendment protection to movies and video games. Human beings, we all agree, have constitutional rights&#8212;but mere machines do not. When the computer in your game console or DVD player &#8220;decides&#8221; to display certain images on a screen, therefore, this is not protected speech, but merely the output of a mechanical process that legislatures may regulate without any special restrictions. All those court rulings that have found these media to be protected forms of expression, therefore, are confused efforts to imbue computers with constitutional rights&#8212;surely foreshadowing the ominous rise of Skynet.
<br /><br />
Probably nobody finds this argument very convincing, and it hardly takes a legal scholar to see what&#8217;s wrong with it: Computers don&#8217;t really autonomously &#8220;decide&#8221; anything: They execute algorithms that embody decisions made by their human programmers. 
</i></blockquote>
Similarly, Paul Levy takes Wu to task by comparing his argument to things like <a href="http://pubcit.typepad.com/clpblog/2012/06/its-not-computers-whose-speech-is-protected-its-their-owners.html" target="_blank">university rankings</a>:
<blockquote><i>
As the alumnus of a college that <a href="http://www.reed.edu/apply/news_and_articles/college_rankings.html" target="_blank">proudly rejects</a> the proposition that the quality of educational institutions can be &#8220;<a href="http://www.usnews.com/education/best-colleges/articles/2011/09/12/how-us-news-calculates-the-college-rankings-2012" target="_blank">measured by a series of data points</a>,&#8221; I will take any opportunity to denigrate the Useless News and World Distort rankings of colleges, law schools and institutions of higher education.&nbsp; But it would never have occurred to me to offer a &#8220;speech by computer&#8221; theory as a basis for denying that the ranking is speech or that it is protected opinion.&nbsp; Maybe a stupid opinion, but that is not a basis for shutting the raters down, or enjoining them to change their rating criteria.&nbsp; Indeed, this theory seems to me absurd &#8212; it is not the computers that have free speech rights, any more than printing presses have free speech rights.&nbsp; It is the media companies that own the printing presses that have free speech rights, and by the same token it is the people and companies who program the computers and publish the results of their calculations that enjoy protection under the First Amendment.
</i></blockquote>
Pretending that Google is a computer that magically generates answers, absent humans regular and consistent input into its algorithm, is a strange position to take, and it really does suggest that we should only protect opinions that don't include a component that uses a computer to analyze the data.  I can't see how that's smart policy or anything close to what was intended in the concept of free speech.<br /><br /><a href="http://www.techdirt.com/articles/20120621/14440519421/speech-via-algorithm-is-still-speech-censoring-it-is-still-censorship.shtml">Permalink</a> | <a href="http://www.techdirt.com/articles/20120621/14440519421/speech-via-algorithm-is-still-speech-censoring-it-is-still-censorship.shtml#comments">Comments</a> | <a href="http://www.techdirt.com/articles/20120621/14440519421/speech-via-algorithm-is-still-speech-censoring-it-is-still-censorship.shtml?op=sharethis">Email This Story</a><br />
 ]]></description>
<slash:department>this-makes-no-sense</slash:department>
<wfw:commentRss>http://www.techdirt.com/comment_rss.php?sid=20120621/14440519421</wfw:commentRss>
</item>
<item>
<pubDate>Thu, 7 Jun 2012 17:00:00 PDT</pubDate>
<title>DailyDirt: Modern Computer Dating</title>
<dc:creator>Michael Ho</dc:creator>
<link>http://www.techdirt.com/articles/20100610/1702589775/dailydirt-modern-computer-dating.shtml</link>
<guid>http://www.techdirt.com/articles/20100610/1702589775/dailydirt-modern-computer-dating.shtml</guid>
<description><![CDATA[ There are search engines for just about everything. Computers help people find all sorts of things faster and easier, so it's not surprising that computer dating is decades old (and <a href="http://www.damninteresting.com/forty-years-of-computer-dating/">started</a> out with punch cards). Technology has gotten a lot better, but has online dating really improved that much?

<ul>
<li> <a title="http://paidcontent.org/2012/02/14/419-why-dating-sites-are-broken/" href="http://bit.ly/JXICGj">Online dating sites claim to be able to find soul mates, but studies suggest they can only provide introductions -- and more often than not, they instill a "shopping mindset" which is not conducive to building lasting relationships.</a> Some psychologists even doubt the possibility of creating algorithms that could successfully match people for real romance. (But experts also scoffed at the first chess-playing computers, too.) [<a href="http://paidcontent.org/2012/02/14/419-why-dating-sites-are-broken/">url</a>]</li>

<li> <a title="http://www.psychologicalscience.org/observer/getArticle.cfm?id=1726" href="http://bit.ly/Ni7YQe">In 2005, Steven Carter, the director of research at eHarmony, described how his work to create predictive models for compatibility became the basis of his dream job.</a> Carter is probably a bit biased, so where is some peer review for eHarmony's research? [<a href="http://www.psychologicalscience.org/observer/getArticle.cfm?id=1726">url</a>]</li>

<li> <a title="http://www.npr.org/blogs/health/2012/04/30/151550273/to-predict-dating-success-the-secrets-in-the-pronouns?" href="http://n.pr/KiNGBS">Perhaps some intense linguistic analysis can predict dating success.</a> Apparently, people who have similar speech patterns are more likely to date each other. Watch your use of pronouns, folks.... [<a href="http://www.npr.org/blogs/health/2012/04/30/151550273/to-predict-dating-success-the-secrets-in-the-pronouns?">url</a>]</li>

<li><b>To discover more interesting tech-related content, <a title="http://www.stumbleupon.com/to/stumble/topic:Technology" href="http://bit.ly/ewIrx5">check out what's currently floating around the StumbleUpon universe.</a></b> [<a href="http://www.stumbleupon.com/to/stumble/topic:Technology">url</a>]  <a title="what's this?" href="#" class="whatsthis help_ddstumble">&nbsp;</a>
</li>
</ul> 


By the way, StumbleUpon can recommend some good <a title="http://www.stumbleupon.com/to/stumble/stumblethru:www.techdirt.com" href="http://bit.ly/fagV8c">Techdirt</a> articles, too.<br /><br /><a href="http://www.techdirt.com/articles/20100610/1702589775/dailydirt-modern-computer-dating.shtml">Permalink</a> | <a href="http://www.techdirt.com/articles/20100610/1702589775/dailydirt-modern-computer-dating.shtml#comments">Comments</a> | <a href="http://www.techdirt.com/articles/20100610/1702589775/dailydirt-modern-computer-dating.shtml?op=sharethis">Email This Story</a><br />
 ]]></description>
<slash:department>urls-we-dig-up</slash:department>
<wfw:commentRss>http://www.techdirt.com/comment_rss.php?sid=20100610/1702589775</wfw:commentRss>
</item>
<item>
<pubDate>Wed, 7 Mar 2012 17:00:00 PST</pubDate>
<title>DailyDirt: Big Brother Ads</title>
<dc:creator>Michael Ho</dc:creator>
<link>http://www.techdirt.com/articles/20120223/14511617855/dailydirt-big-brother-ads.shtml</link>
<guid>http://www.techdirt.com/articles/20120223/14511617855/dailydirt-big-brother-ads.shtml</guid>
<description><![CDATA[ The year 1984 didn't quite turn out the way the book did, but we're already living in a world where there are ubiquitous cameras tracking our every move. But it's not necessarily the government behind the cameras -- a lot of cameras are for private security and... possibly advertisers. Here are a few stories that might make you want to dress up like the <a href="http://upload.wikimedia.org/wikipedia/commons/9/9e/Unabomber-sketch.png">Unabomber</a> when you go out. 

<ul>
<li> <a title="http://www.theatlantic.com/technology/archive/2012/02/the-bus-stop-that-knows-youre-a-lady/253365/" href="http://bit.ly/wr5dgU">A billboard at a London bus stop determines (or tries to) people's gender and present ads accordingly.</a> The system apparently is about 90% accurate, and it doesn't apologize for its mistakes. [<a href="http://www.theatlantic.com/technology/archive/2012/02/the-bus-stop-that-knows-youre-a-lady/253365/">url</a>]</li>

<li> <a title="http://www.wired.com/gadgets/miscellaneous/news/2007/06/eyetracking" href="http://bit.ly/zBRTs0">Eyetracking billboards haven't exactly caught on, but the technology exists with cameras that can tell when your retinas are aligned for optimal ad delivery.</a> If these eye-trackers do catch on, there might be a new market of ad-blocking sunglasses... [<a href="http://www.wired.com/gadgets/miscellaneous/news/2007/06/eyetracking">url</a>]</li>

<li> <a title="http://money.cnn.com/2011/04/14/technology/immersive_labs_targeted_ads/index.htm" href="http://cnnmon.ie/A1uav2">Immersive Labs has a prototype ad system that tries to personalize and target its promotions using a collection of software tools that include face recognition and machine learning to figure out the demographics of its audience.</a> The creators of this system say their product will only deliver the ads that people want to see. [Ahem.] [<a href="http://money.cnn.com/2011/04/14/technology/immersive_labs_targeted_ads/index.htm">url</a>]</li>

<li><b>To discover more interesting advertising-related content, <a title="http://www.stumbleupon.com/to/stumble/topic:448" href="http://bit.ly/osqk34">check out what's floating around on StumbleUpon.</a></b> [<a href="http://www.stumbleupon.com/to/stumble/topic:481">url</a>]  <a title="what's this?" href="#" class="whatsthis help_ddstumble">&nbsp;</a>
</li>
</ul> 

By the way, StumbleUpon can recommend some good <a title="http://www.stumbleupon.com/to/stumble/stumblethru:www.techdirt.com" href="http://bit.ly/fagV8c">Techdirt</a> articles, too.<br /><br /><a href="http://www.techdirt.com/articles/20120223/14511617855/dailydirt-big-brother-ads.shtml">Permalink</a> | <a href="http://www.techdirt.com/articles/20120223/14511617855/dailydirt-big-brother-ads.shtml#comments">Comments</a> | <a href="http://www.techdirt.com/articles/20120223/14511617855/dailydirt-big-brother-ads.shtml?op=sharethis">Email This Story</a><br />
 ]]></description>
<slash:department>urls-we-dig-up</slash:department>
<wfw:commentRss>http://www.techdirt.com/comment_rss.php?sid=20120223/14511617855</wfw:commentRss>
</item>
<item>
<pubDate>Mon, 25 Apr 2011 12:13:07 PDT</pubDate>
<title>The Infinite Loop Of Algorithmic Pricing On Amazon... Or How A Book On Flies Cost $23,698,655.93</title>
<dc:creator>Mike Masnick</dc:creator>
<link>http://www.techdirt.com/articles/20110425/03522114026/infinite-loop-algorithmic-pricing-amazon-how-book-flies-cost-2369865593.shtml</link>
<guid>http://www.techdirt.com/articles/20110425/03522114026/infinite-loop-algorithmic-pricing-amazon-how-book-flies-cost-2369865593.shtml</guid>
<description><![CDATA[ <a href="https://twitter.com/#!/glynmoody/statuses/61867629597564928" target="_blank">Glyn Moody</a> points us to an amusing story by biologist Michael Eisen, about two Amazon book sellers who (it appears) each used algorithmic pricing to set their own prices on books off of what the other one was pricing it at, leading to an <a href="http://www.michaeleisen.org/blog/?p=358" target="_blank">ever escalating price on this particular book about flies</a>.  It started with someone looking up a copy of the book <i>The Making of a Fly</i> on Amazon to order a copy for Eisen's research lab at Berkeley and noticing the odd pricing on a couple of used books:
<center>
<img src="http://i.imgur.com/yvrYd.png" width=560 />
</center>
And while everyone at first assumed a typo or some other mistake, they soon discovered it was likely something else:
<blockquote><i>
Amazingly, when I reloaded the page the next day, both priced had gone UP! Each was now nearly $2.8 million. And whereas previously the prices were $400,000 apart, they were now within $5,000 of each other. Now I was intrigued, and I started to follow the page incessantly. By the end of the day the higher priced copy had gone up again. This time to $3,536,675.57. And now a pattern was emerging.
<br /><br />
On the day we discovered the million dollar prices, the copy offered by <a href="http://www.amazon.com/gp/help/seller/at-a-glance.html/ref=olp_merch_name_2?ie=UTF8&#038;isAmazonFulfilled=0&#038;asin=0632030488&#038;marketplaceSeller=0&#038;seller=A2Z2CSK1PH6GOI">bordeebook</a> was1.270589 times the price of the copy offered by <a href="http://www.amazon.com/gp/help/seller/at-a-glance.html/ref=olp_merch_name_1?ie=UTF8&#038;isAmazonFulfilled=0&#038;asin=0632030488&#038;marketplaceSeller=0&#038;seller=A2E1ZC9J7JOJBQ">profnath</a>. And now the bordeebook copy was 1.270589 times profnath again. So clearly at least one of the sellers was setting their price algorithmically in response to changes in the other&rsquo;s price. I continued to watch carefully and the full pattern emerged.
<br /><br />
Once a day profnath&nbsp;set their price to be 0.9983 times bordeebook's price. The prices would remain close for several hours, until bordeebook "noticed" profnath's change and elevated their price to 1.270589 times profnath's higher price. The pattern continued perfectly for the next week.
</i></blockquote>
In other words, by basing the price on a specific multiple of each other's price, they created something of an infinite loop of automated pricing.  Of course, since it only seemed to check once a day, the loop didn't spiral totally out of control (well, depending on your opinion of the value of a book on flies).  Apparently, the escalation in price went on for another week and a half or so before someone finally noticed... but not before the book got all the way up to $23,698,655.93:
<center>
<img src="http://i.imgur.com/E8SEj.png" width=560 />
</center>
The book is apparently now available at a much cheaper "bargain" price, though you may want to pick up your copy for resale in the future if the "value" keeps appreciating.<br /><br /><a href="http://www.techdirt.com/articles/20110425/03522114026/infinite-loop-algorithmic-pricing-amazon-how-book-flies-cost-2369865593.shtml">Permalink</a> | <a href="http://www.techdirt.com/articles/20110425/03522114026/infinite-loop-algorithmic-pricing-amazon-how-book-flies-cost-2369865593.shtml#comments">Comments</a> | <a href="http://www.techdirt.com/articles/20110425/03522114026/infinite-loop-algorithmic-pricing-amazon-how-book-flies-cost-2369865593.shtml?op=sharethis">Email This Story</a><br />
 ]]></description>
<slash:department>do-they-have-a-kindle-version?</slash:department>
<wfw:commentRss>http://www.techdirt.com/comment_rss.php?sid=20110425/03522114026</wfw:commentRss>
</item>
<item>
<pubDate>Tue, 1 Feb 2011 12:10:00 PST</pubDate>
<title>Google's Childish Response To Microsoft Using Google To Increase Bing Relevance</title>
<dc:creator>Mike Masnick</dc:creator>
<link>http://www.techdirt.com/articles/20110201/11022312911/googles-childish-response-to-microsoft-using-google-to-increase-bing-relevance.shtml</link>
<guid>http://www.techdirt.com/articles/20110201/11022312911/googles-childish-response-to-microsoft-using-google-to-increase-bing-relevance.shtml</guid>
<description><![CDATA[ It's inevitable as a company gets bigger and older that rather than just competing in the market, it starts attacking competitors and accusing them of doing something "wrong."  It's too bad that Google appears to have reached this stage.  There have been plenty of stories lately about Google's decreasing relevance and how its search results have been getting worse.  There are plenty of ways to respond to this and improving search quality should be the main focus.  But it looks like Google has, instead, decided to call out competitors.  Specifically, Google set up an elaborate and pointless "sting operation," which appears to show <a href="http://searchengineland.com/google-bing-is-cheating-copying-our-search-results-62914" target="_blank">that Microsoft uses Google results as a part of its overall relevance algorithm</a>.  Basically, it looks like for users who have the Bing toolbar installed, Microsoft aggregates some search information, perhaps including Google results, and weighs them (only partially) into its own algorithm.
<br /><br />
This seems like a perfectly reasonable thing to do.  Google's search results are public and as an established player in the market, almost every comparison of alternative search engines, including Bing, compares it to Google.  So, making use of Google data to improve its own rankings seems like a rather smart move.
<br /><br />
Remember, too, that Google's own search algorithm is based on viewing what people are doing online and coming up with a ranking based on that.  How is that any different than Microsoft viewing a variety of information online -- including Google's own search rankings -- and using that as the basis of its own rankings?  But instead of recognizing that this is all perfectly reasonable, Google starts acting like the RIAA, accusing Microsoft of "cheating" and doing something that is potentially illegal.  It even pops out this line from Amit Singhal, a Google Fellow who apparently oversees Google's search ranking algorithm.
<blockquote><i>
"I've got no problem with a competitor developing an innovative algorithm. But copying is not innovation, in my book."
</i></blockquote>
As if Google hasn't copied the work of others in the past?  The very basis for the original Page Rank was "copied" from Jon Kleinberg's research and then built upon that work.  It was not a direct copy, just as Microsoft's search results are not a direct copy.  For Google to attack a competitor for using open information on the web -- the same way it does -- seems like the height of hypocrisy.  It's fine for Google to crawl and index whatever sites it wants in order to set up its ranking algorithms, but the second someone looks at Google's own rankings as part of their own determination, suddenly its "cheating"? 
<br /><br />
This seems like the latest in a series of indications that Google has moved past the innovation stage into the "protecting its turf" stage.  That would be a shame.<br /><br /><a href="http://www.techdirt.com/articles/20110201/11022312911/googles-childish-response-to-microsoft-using-google-to-increase-bing-relevance.shtml">Permalink</a> | <a href="http://www.techdirt.com/articles/20110201/11022312911/googles-childish-response-to-microsoft-using-google-to-increase-bing-relevance.shtml#comments">Comments</a> | <a href="http://www.techdirt.com/articles/20110201/11022312911/googles-childish-response-to-microsoft-using-google-to-increase-bing-relevance.shtml?op=sharethis">Email This Story</a><br />
 ]]></description>
<slash:department>get-over-it</slash:department>
<wfw:commentRss>http://www.techdirt.com/comment_rss.php?sid=20110201/11022312911</wfw:commentRss>
</item>
<item>
<pubDate>Wed, 20 Oct 2010 06:37:48 PDT</pubDate>
<title>Traders Convicted For Figuring Out Auto Trading Algorithm; How Is That Illegal?</title>
<dc:creator>Mike Masnick</dc:creator>
<link>http://www.techdirt.com/articles/20101015/01330611439/traders-convicted-for-figuring-out-auto-trading-algorithm-how-is-that-illegal.shtml</link>
<guid>http://www.techdirt.com/articles/20101015/01330611439/traders-convicted-for-figuring-out-auto-trading-algorithm-how-is-that-illegal.shtml</guid>
<description><![CDATA[ <a href="http://news.slashdot.org/story/10/10/15/0139217/Norwegian-Day-Traders-Convicted-For-Manipulating-Computer-Trading-System?from=twitter" target="_blank">Slashdot</a> points us to the story of how two Norwegian day traders have been convicted and given suspended jail sentences for <a href="http://www.computerworlduk.com/news/security/3244186/norwegian-traders-convicted-for-outsmarting-us-stock-broker-algorithm/" target="_blank">outsmarting an automated computer trading system</a>, enabling them to make money.  The details are not entirely clear, but from what's in the article, it sounds like they observed some patterns in the way the system responded to certain trades, and then they took advantage of that.  Of course, that's exactly what automated computer trading systems, themselves, are supposed to do.  They're supposed to notice patterns in trading and take advantage of that.  So, would it have been illegal for the same automated trading system to notice patterns in certain human trades and take advantage of it?<br /><br /><a href="http://www.techdirt.com/articles/20101015/01330611439/traders-convicted-for-figuring-out-auto-trading-algorithm-how-is-that-illegal.shtml">Permalink</a> | <a href="http://www.techdirt.com/articles/20101015/01330611439/traders-convicted-for-figuring-out-auto-trading-algorithm-how-is-that-illegal.shtml#comments">Comments</a> | <a href="http://www.techdirt.com/articles/20101015/01330611439/traders-convicted-for-figuring-out-auto-trading-algorithm-how-is-that-illegal.shtml?op=sharethis">Email This Story</a><br />
 ]]></description>
<slash:department>no-whammies</slash:department>
<wfw:commentRss>http://www.techdirt.com/comment_rss.php?sid=20101015/01330611439</wfw:commentRss>
</item>
<item>
<pubDate>Fri, 26 Feb 2010 09:06:00 PST</pubDate>
<title>Yes, Google Will Even Delete Its Own Employees' Sites From Google Index If They Screw Up</title>
<dc:creator>Mike Masnick</dc:creator>
<link>http://www.techdirt.com/articles/20100224/0138408282.shtml</link>
<guid>http://www.techdirt.com/articles/20100224/0138408282.shtml</guid>
<description><![CDATA[ Over the years, we've seen plenty of people get mad and <a href="http://www.techdirt.com/articles/20060607/0334227.shtml">sue Google</a> over getting either dropped from Google's index, or simply ranked really low on certain searches.  And, of course, for some there are always totally unsubstantiated <a href="http://www.techdirt.com/articles/20090217/1841023807.shtml">conspiracy theories</a> about Google purposely "punishing" a company or individual it doesn't like.  To date, however, there's been no evidence at all that Google acts in such a capricious manner when it comes to rankings.  If the company did so, and it ever came out, the hit to Google's reputation would be something fierce.  It's even more ridiculous when you consider that all of the accusations of such personal attacks seem to come from tiny companies -- hardly any kind of threat to Google, anyway.
<br /><br />
That said, if you want even more evidence that Google's ranking decisions aren't personal, but actually are based on what its system feels will give the best possible results, witness the story of <i>Google employee</i> Jason Morrison, who <a href="http://www.jasonmorrison.net/content/2010/how-my-site-disappeared-from-google-search/" target="_blank">recently discovered that his <i>own personal site</i> had be delisted from Google</a>.  It actually took him a few weeks to notice this, but once he did, and dug into the issue (using Google's public tool and his own site's admin tools) he quickly realized that he had made a mistake that caused Google's crawlers to believe that his site was no longer up.
<br /><br />
Now, that certainly doesn't preclude the possibility that Google takes revenge on sites it doesn't like, but it's at least more evidence that the ranking system really is pretty algorithmically focused -- and even Google employees aren't immune to being delisted for screwing up.  If your site gets delisted from Google, it's not personal.<br /><br /><a href="http://www.techdirt.com/articles/20100224/0138408282.shtml">Permalink</a> | <a href="http://www.techdirt.com/articles/20100224/0138408282.shtml#comments">Comments</a> | <a href="http://www.techdirt.com/articles/20100224/0138408282.shtml?op=sharethis">Email This Story</a><br />
 ]]></description>
<slash:department>it-ain't-personal</slash:department>
<wfw:commentRss>http://www.techdirt.com/comment_rss.php?sid=20100224/0138408282</wfw:commentRss>
</item>
<item>
<pubDate>Fri, 20 Nov 2009 01:49:13 PST</pubDate>
<title>Winner Takes All, Long Tails And The Fractilization Of Culture</title>
<dc:creator>Mike Masnick</dc:creator>
<link>http://www.techdirt.com/articles/20091106/1827296844.shtml</link>
<guid>http://www.techdirt.com/articles/20091106/1827296844.shtml</guid>
<description><![CDATA[ Reader Eileen points us to a thought-provoking article by Joshua-Michele Ross discussing the idea that, rather than a diverse "long-tail" culture, we're actually being driven to a <a href="http://radar.oreilly.com/2009/11/three-paradoxes-of-the-interne.html" target="_blank">homogenized "winner-take-all" culture</a> thanks to the rise of our robot overlords, better known as online recommendation engines.  Or something like that.  It's a nice theory, with some <a href="http://whimsley.typepad.com/whimsley/2009/03/online-monoculture-and-the-end-of-the-niche.html" target="_blank">interesting statistical modelling</a> behind it.  And, I've always been interested in "winner takes all" economies, since the guy who taught me Econ 101 literally <a href="http://books.google.com/books?id=nWFtL7WS5WwC&#038;q=winner+take+all,+robert+frank&#038;dq=winner+take+all,+robert+frank&#038;ei=dPT9Sv2yOZPKywT19OjVAg" target="_blank">wrote the book</a> on "winner takes all" economics.
<br /><br />
That said, I think this really only tells a part of the story -- and maybe not the most important or most interesting part.  That's because (and, again, this may be due to my own econ education) it doesn't surprise me in the slightest that we'd see hits follow a winner takes all approach (that's how hits work).  Nor is it a surprise that the effect would seem stronger as the world globalizes and borders and barriers become less of an issue.  So, yes, of course there will be a "globalized" winner takes all situation at the hits level.  But is that all?
<br /><br />
What's much more interesting to me is what happens <i>beyond</i> the hits.  And, as you start to dig down into subsectors or subcultures, you begin to notice an interesting pattern there as well: that those subsectors and subcultures follow that same power law pattern themselves.  The big name bands in a subculture may seem "small" in the wider world, but they're huge within the subculture.  Within that subculture, they're the winner who took all -- but from a more limited population.
<br /><br />
In some ways, it's the <b>fractalization</b> of culture.  
<br /><br />
Just as a fractal repeats its same pattern as you zoom in and look closer on the smaller segments, so do cultural subsegments.  And those segments continue to thrive, despite the recommendation systems just pushing people to the hits.  Part of that may be that once you've begun exploring those subcultures, the recommendation engines and collaborative filters drive you towards the "hits within" the subculture -- or it may be that the impact of algorithmic recommendation engines isn't quite as dominating as some make it out to be.  Yes, people do rely on those recommendation engines... somewhat. But they trust people they know even more.  And once you get involved in a subculture you quickly find other people already involved in that culture who act as guides who point you both to the "hits" but also to the interesting and "diverse" long tail places to go as well.
<br /><br />
So, yes, there is a winner take all effect found in the recommendation engines, but it hasn't resulted in less diversity within our cultural output or our cultural consumption -- and that's because people don't just follow that limited algorithmic overlord to find the content they want to consume.  In fact, the original statistical model highlighted above more or less makes this point.  Basically, it shows that even if each individual sees a more diverse culture, it can still end up with a more homogenized culture -- but really only among the hits.  Basically, because the world is global, the really big hits go global and become winner-take-all in a much larger market.  But, at the same time, the niches thrive as well.<br /><br /><a href="http://www.techdirt.com/articles/20091106/1827296844.shtml">Permalink</a> | <a href="http://www.techdirt.com/articles/20091106/1827296844.shtml#comments">Comments</a> | <a href="http://www.techdirt.com/articles/20091106/1827296844.shtml?op=sharethis">Email This Story</a><br />
 ]]></description>
<slash:department>rethinking-the-niche</slash:department>
<wfw:commentRss>http://www.techdirt.com/comment_rss.php?sid=20091106/1827296844</wfw:commentRss>
</item>
<item>
<pubDate>Fri, 31 Jul 2009 07:07:00 PDT</pubDate>
<title>Can You Copyright Algorithmic Output?</title>
<dc:creator>Mike Masnick</dc:creator>
<link>http://www.techdirt.com/articles/20090730/1747055719.shtml</link>
<guid>http://www.techdirt.com/articles/20090730/1747055719.shtml</guid>
<description><![CDATA[ A bunch of folks have been sending in Neil McAllister's writeup at InfoWorld about how Wolfram Alpha, the incredibly overhyped "knowledge engine" (that, in my experience doesn't work very well) is <a href="http://infoworld.com/print/85248" target="_new">claiming copyright on all of its output</a>, which raises questions about what would happen if others did the same thing:
<blockquote><i>
In other words, Wolfram Research is claiming that each page of results returned by the Wolfram Alpha engine is a unique, copyrightable work, like a report or term paper. That makes Wolfram Alpha different not just from classic search engines, but from most software. While software companies routinely retain sole ownership of their software and license it to users, Wolfram Research has taken the additional step of claiming ownership of the output of the software itself. It's a bold assertion, and one that could have significant ramifications for the software industry as a whole.
</i></blockquote>
It really depends on the output, but in many cases I have trouble believing the output really is copyrightable.  After all, you cannot copyright facts and (in the US, at least) you can't copyright a collection of facts, either.  The article doesn't discuss that, and seems to assume that the output may be copyrightable, but I would think that it would need to be significantly more unique and have additional creativity before it could be covered (and then, only the unique parts would be covered).  Still, there may be a legal gray area, as McAllister notes:
<blockquote><i>
Suppose you have an Excel spreadsheet full of numbers that you input, but then you ask Excel to generate a series of complex graphs based on rules, formulae, and templates designed by Microsoft. Or what about pivot tables? What about mash-ups or tools like Mozilla Jetpack? If unique presentations based on software-based manipulation of mundane data are copyrightable, who retains what rights to the resulting works?
</i></blockquote>
I'm guessing that the graphs still wouldn't be copyrightable, as they'd really just be the same collection of data, but you could see a mathematically illiterate court finding otherwise...<br /><br /><a href="http://www.techdirt.com/articles/20090730/1747055719.shtml">Permalink</a> | <a href="http://www.techdirt.com/articles/20090730/1747055719.shtml#comments">Comments</a> | <a href="http://www.techdirt.com/articles/20090730/1747055719.shtml?op=sharethis">Email This Story</a><br />
 ]]></description>
<slash:department>do-computers-need-incentive-to-create?</slash:department>
<wfw:commentRss>http://www.techdirt.com/comment_rss.php?sid=20090730/1747055719</wfw:commentRss>
</item>
<item>
<pubDate>Fri, 24 Jul 2009 10:29:49 PDT</pubDate>
<title>Two Separate Rulings In France Split Over Whether Google's Suggestion Algorithm Can Be Libelous</title>
<dc:creator>Mike Masnick</dc:creator>
<link>http://www.techdirt.com/articles/20090724/0407145647.shtml</link>
<guid>http://www.techdirt.com/articles/20090724/0407145647.shtml</guid>
<description><![CDATA[ Reader Yann alerts us to an interesting set of lawsuits and decisions in France, both concerning the Google Suggest feature.  One case involved a company named <a href="http://translate.google.com/translate?u=http%3A%2F%2Fwww.pcinpact.com%2Factu%2Fnews%2F51967-google-suggest-direct-energie-suggestion.htm&#038;sl=fr&#038;tl=en&#038;hl=en&#038;ie=UTF-8" target="_new">Direct Energie</a> and the other with a company named <a href="http://translate.google.com/translate?u=http%3A%2F%2Fwww.pcinpact.com%2Factu%2Fnews%2F52001-cnfdi-google-suggest-arnaque-injure.htm&#038;sl=fr&#038;tl=en&#038;hl=en&#038;ie=UTF-8" target="_new">CNFDI</a> (both links to the Google translation of the news).
<br /><br />
In both cases, the companies were upset that when people started searching on their company names, the first suggestion was their company name followed by the word "arnaque," which means "scam."  Of course, as you probably know, Google Suggest works by finding the most common searches on what you've typed and letting you know.  So, all this really meant was that an awful lot of people were doing searches questioning whether or not these two companies were scams.  But, is Google liable for its algorithm <i>accurately</i> suggesting the most common searches associated with those company names?  It appears the courts split on that decision (it's worth noting that there was one major difference between the lawsuits: Direct Energie sued under civil code, while CNFDI sued for libel -- which apparently makes it a criminal case in France.
<br /><br />
With Direct Energie, the judge seemed to not really understand Google Suggest or how it worked, declaring that no algorithm could justify the prejudice caused by Google.  He then got confused, saying that it was clearly Google's fault because the search on "direct energie arnaque" was not the first alphabetically in the list, nor did it have the highest number of results.  Despite it being explained by Google, the judge seems to have totally ignored the reason why it was at the top of the list (the number of people searching for it).  Because of this, he said it's no limit on free speech to force Google to change the results, and ordered Google to do so (though, did not allow for any damages to be awarded).  This seems to get the basic facts backwards, and it seems quite ridiculous to find Google guilty of such a charge when all its actually doing is <i>accurately</i> counting up what people are legitimately searching for.
<br /><br />
The CNFDI ruling, seems much more reasonable.  There was one oddity (though it's probably got more to do with French law than with the judge), and that is that the judge ruled that Google <i>could</i> be liable for libel because the company had been informed by CNFDI of the issue, thereby removing any safe harbors.  In the US, Section 230 safe harbors on libel <i>thankfully</i> do not get waived if you've been informed.  Instead, they take the much more logical position that a third party service provider should never be blamed for actions of its users.  Thus, it would be flat-out ridiculous to blame Google for the phrases people are searching for.  But, even having lost its local "safe harbor" protections, the judge properly recognized that the suggestion came from the algorithm looking at what people were searching for, and noted that the suggestion was based on "a valid observation."  On top of that, he pointed out that search engines are "important tools for the free circulation of ideas and information," and the fact that many people were questioning whether CNFDI was a scam was, in fact, important and potentially useful information, and thus not libelous by itself.  Finally, the court also noted that forcing Google to remove such a suggestion would be too big a burden on free speech and citizens' rights.
<br /><br />
It should be no surprise that I think the second ruling is much more sensible, while the first ruling makes little sense, and appears to have been decided without a full understanding of what Google's Suggest feature is or how it works.  Still, I imagine we'll be seeing similar cases around the world... and hopefully they'll find themselves in front of judges more like the one that dealt with the CNFDI case...<br /><br /><a href="http://www.techdirt.com/articles/20090724/0407145647.shtml">Permalink</a> | <a href="http://www.techdirt.com/articles/20090724/0407145647.shtml#comments">Comments</a> | <a href="http://www.techdirt.com/articles/20090724/0407145647.shtml?op=sharethis">Email This Story</a><br />
 ]]></description>
<slash:department>confusion-abounds</slash:department>
<wfw:commentRss>http://www.techdirt.com/comment_rss.php?sid=20090724/0407145647</wfw:commentRss>
</item>
<item>
<pubDate>Tue, 14 Jul 2009 20:22:00 PDT</pubDate>
<title>Isn't There Something Ironic In An Anonymous Exec Demanding Transparency From Google?</title>
<dc:creator>Mike Masnick</dc:creator>
<link>http://www.techdirt.com/articles/20090714/0434585543.shtml</link>
<guid>http://www.techdirt.com/articles/20090714/0434585543.shtml</guid>
<description><![CDATA[ It really is amazing sometimes to see how many people think that Google "owes" them something.  For example, we've had a few different stories about companies <a href="http://www.techdirt.com/articles/20061027/192602.shtml">suing Google</a> because they don't like how Google ranks them.  That makes little sense.  Google doesn't owe anyone a spot in its index.  It determines its index by figuring out what it thinks people will like best, and it's always tweaking it.  If it fails to figure that out properly and someone else (like Microsoft) does figure it out, then Google will lose business.  So, it seems a bit odd that some anonymous "well known exec at one of the largest sites on the Internet" is suddenly <a href="http://www.techcrunch.com/2009/07/13/the-time-has-come-to-regulate-search-engine-marketing-and-seo/" target="_new">demanding transparency into how Google ranks content</a>, suggesting that it's somehow unfair and arbitrary in its rankings -- and only by opening up the details of its algorithm will "fairness" be restored.
<br /><br />
<a href="http://www.dotcult.com">Ryan</a>, who alerted us to this story, has written up a <a href="
http://www.dotcult.com/the-time-has-come-to-regulate-techcrunch" target="_new">biting, but reasonable, response</a>, where he notes that being ranked highly in Google is no one's right.  And demanding that Google be transparent about its algorithm is meaningless (while being especially ironic, given that this "well-known exec" is demanding transparency while wanting to remain anonymous himself).  The key point Ryan makes:
<blockquote><i>
You want an algorithm, here it is:
<br />1.) Sites that are useful to visitors will rank high.
<br />2.) Popular sites that are useful to visitors will rank higher.
<br />3.) Sites that don't offer any value to the web or are irrelevant to the query won't rank well.
<br />4.) Sites that are harmful or spammy won't be included in the index.
<br /><br />
Seriously, that's Google’s algorithm in plain English. There's your disclosure. The weighting factors and code behind it don't matter -- these principles are all you really need to know.
</i></blockquote>
Indeed.  Create useful sites with useful content that people use, and don't be spammy, and you'll most likely rank well in Google.  You don't need to force Google to reveal the nuts and bolts of its algorithm.  That doesn't change anything.  If you're trying to craft your websites to the specifics of the algorithm, you're already lost.  If you're creating websites that match the "plain English" code above, you're going to be just fine.<br /><br /><a href="http://www.techdirt.com/articles/20090714/0434585543.shtml">Permalink</a> | <a href="http://www.techdirt.com/articles/20090714/0434585543.shtml#comments">Comments</a> | <a href="http://www.techdirt.com/articles/20090714/0434585543.shtml?op=sharethis">Email This Story</a><br />
 ]]></description>
<slash:department>entitlement-culture</slash:department>
<wfw:commentRss>http://www.techdirt.com/comment_rss.php?sid=20090714/0434585543</wfw:commentRss>
</item>
<item>
<pubDate>Fri, 19 Sep 2008 07:55:00 PDT</pubDate>
<title>Garbage In, Financial Crisis Out</title>
<dc:creator>Mike Masnick</dc:creator>
<link>http://www.techdirt.com/articles/20080918/1826002310.shtml</link>
<guid>http://www.techdirt.com/articles/20080918/1826002310.shtml</guid>
<description><![CDATA[ With everyone trying to figure out just what went wrong to cause the rather spectacular financial mess Wall Street finds itself in these days, Saul Hansell over at the NY Times wanted to find out why all the sophisticated risk management quant algorithms that Wall St. has been so big on lately failed to warn of impending doom.  His answer, basically, is that <a href="http://bits.blogs.nytimes.com/2008/09/18/how-wall-streets-quants-lied-to-their-computers/?partner=rssuserland&#038;emc=rss" target="_new">people on Wall St. were lying to the algorithms</a>, coming up with ways to purposely enter data such that the risk seemed much less than it actually was -- in order to let them keep pushing the boundary.  Then, it became a situation where people start relying on the computers just because the <a href="http://www.techdirt.com/articles/20050220/2340207_F.shtml">computer says so</a> -- even though the data is bad.  This happens time and time again.  Even when people know that computers make mistakes, it's just so convenient to have a computer "confirm" your thinking that you start ignoring other warning signs.<br /><br /><a href="http://www.techdirt.com/articles/20080918/1826002310.shtml">Permalink</a> | <a href="http://www.techdirt.com/articles/20080918/1826002310.shtml#comments">Comments</a> | <a href="http://www.techdirt.com/articles/20080918/1826002310.shtml?op=sharethis">Email This Story</a><br />
 ]]></description>
<slash:department>so-much-for-the-quants</slash:department>
<wfw:commentRss>http://www.techdirt.com/comment_rss.php?sid=20080918/1826002310</wfw:commentRss>
</item>
<item>
<pubDate>Tue, 21 Aug 2007 17:36:00 PDT</pubDate>
<title>From Hedge Funds To Skype, Collapses Prove Unavoidable</title>
<dc:creator>Joseph Weisenthal</dc:creator>
<link>http://www.techdirt.com/articles/20070821/105147.shtml</link>
<guid>http://www.techdirt.com/articles/20070821/105147.shtml</guid>
<description><![CDATA[ Is there a connection between the <a href="http://www.techdirt.com/articles/20070809/121909.shtml">recent meltdown at quant funds</a> and <a href="http://www.techdirt.com/articles/20070816/102346.shtml">last week's outage at Skype</a>?  Nick Carr makes the provocative argument that both events are the result of what happens when <a href="http://www.roughtype.com/archives/2007/08/skype_and_the_h.php">algorithms fail to anticipate behavior</a> that is somehow out of the ordinary.  In the case of quant funds, their models failed to anticipate the market's wild volatility, whereas with Skype (if you believe the company's official explanation), the glitch was the result of mass reboots taxing network capacity. Interestingly, both <a href="http://heartbeat.skype.com/2007/08/the_microsoft_connection_explained.html">Skype engineers</a> and <a href="http://www.bloomberg.com/apps/news?pid=20601087&#038;sid=a3xa5ABlNlbA&#038;refer=home">hedge fund managers</a> were heard using the phrase "perfect storm" to describe the sequence of events that lead to their respective collapses.  Of course, as hedge funds learn every few years, these perfect storms that are mathematically supposed to occur just once in a thousand years, seem to happen quite a bit more often.  The same goes for any network that suffers an outage despite the best laid contingency plans.  The problem is that it's difficult to craft an algorithm or a model that's robust during 'normal' times and abnormal times.  In finance, one hopes that the profits are big enough during the good so that you can survive the occasional mess.  The one problem, of course, with the comparison between hedge funds and Skype is that Skype's explanation doesn't ring particularly true.  The connection between Microsoft patches, mass reboots and the network collapse <a href="http://www.techdirt.com/articles/20070820/163914.shtml">seems tenuous at best</a>.  Thus, it's entirely possible that this particularly outage had nothing to do with abnormal crowd behavior.  Still, as the <a href="http://www.techdirt.com/articles/20070724/150454.shtml">surprise outage at 365 Main</a> demonstrates, it's difficult, if not fully impossible, to completely inoculate oneself against adverse events.<br /><br /><a href="http://www.techdirt.com/articles/20070821/105147.shtml">Permalink</a> | <a href="http://www.techdirt.com/articles/20070821/105147.shtml#comments">Comments</a> | <a href="http://www.techdirt.com/articles/20070821/105147.shtml?op=sharethis">Email This Story</a><br />
 ]]></description>
<slash:department>crashing-down</slash:department>
<wfw:commentRss>http://www.techdirt.com/comment_rss.php?sid=20070821/105147</wfw:commentRss>
</item>
</channel>
</rss>