No, A 'Supercomputer' Did NOT Pass The Turing Test For The First Time And Everyone Should Know Better
from the what-a-waste-of-time dept
The 65 year-old iconic Turing Test was passed for the very first time by supercomputer Eugene Goostman during Turing Test 2014 held at the renowned Royal Society in London on Saturday.Okay, almost everything about the story is bogus. Let's dig in:
'Eugene', a computer programme that simulates a 13 year old boy, was developed in Saint Petersburg, Russia. The development team includes Eugene's creator Vladimir Veselov, who was born in Russia and now lives in the United States, and Ukrainian born Eugene Demchenko who now lives in Russia.
[....] If a computer is mistaken for a human more than 30% of the time during a series of five minute keyboard conversations it passes the test. No computer has ever achieved this, until now. Eugene managed to convince 33% of the human judges that it was human.
- It's not a "supercomputer," it's a chatbot. It's a script made to mimic human conversation. There is no intelligence, artificial or not involved. It's just a chatbot.
- Plenty of other chatbots have similarly claimed to have "passed" the Turing test in the past (often with higher ratings). Here's a story from three years ago about another bot, Cleverbot, "passing" the Turing Test by convincing 59% of judges it was human (much higher than the 33% Eugene Goostman) claims.
- It "beat" the Turing test here by "gaming" the rules -- by telling people the computer was a 13-year-old boy from Ukraine in order to mentally explain away odd responses.
- The "rules" of the Turing test always seem to change. Hell, Turing's original test was quite different anyway.
- As Chris Dixon points out, you don't get to run a single test with judges that you picked and declare you accomplished something. That's just not how it's done. If someone claimed to have created nuclear fusion or cured cancer, you'd wait for some peer review and repeat tests under other circumstances before buying it, right?
- The whole concept of the Turing Test itself is kind of a joke. While it's fun to think about, creating a chatbot that can fool humans is not really the same thing as creating artificial intelligence. Many in the AI world look on the Turing Test as a needless distraction.
Basically, any reporter should view extraordinary claims associated with Warwick with extreme caution. But that's not what happened at all. Instead, as is all too typical with Warwick claims, the press went nutty over it, including publications that should know better. Here are just a few sample headlines. The absolute worst are the ones who claim this is a "supercomputer."
- The Verge: Computer passes Turing Test for first time by convincing judges it is a 13-year-old boy
- Venture Beat: Talk to the computer that passed the Turing Test, a historic artificial intelligence milestone
- Yahoo Tech: Turing Test Bested, Robot Overlords Creep Closer
- NBC News: Turing Test: Computer Program Convinces Judges It's Human
- Washington Post: A computer just passed the Turing Test in landmark trial
- The Independent: Turing Test breakthrough as super-computer becomes first to convince us it's human
- PC World: An AI milestone: Chatbot passes Turing Test by posing as 13-year-old boy
- The Wire: For the First Time Ever, a Computer Passed Turing Test for Artificial Intelligence
- Gizmodo: A Computer Program Has Passed the Turing Test For the First Time
- ZDNet: Computer chatbot 'Eugene Goostman' passes the Turing test
- Ars Technica: Eugene—the supercomputer, not 13-year-old—first to beat the Turing Test
- The Guardian: Computer simulating 13-year-old boy becomes first to pass Turing test
- CNET: Computer fools humans, passes Turing Test
- Computerworld: Supercomputer passes Turing Test by posing as a teenager
- Science Alert: Meet the first computer to pass the Turing Test