No, A 'Supercomputer' Did NOT Pass The Turing Test For The First Time And Everyone Should Know Better

from the what-a-waste-of-time dept

So, this weekend's news in the tech world was flooded with a "story" about how a "chatbot" passed the Turing Test for "the first time," with lots of publications buying every point in the story and talking about what a big deal it was. Except, almost everything about the story is bogus and a bunch of gullible reporters ran with it, because that's what they do. First, here's the press release from the University of Reading, which should have set off all sorts of alarm bells for any reporter. Here are some quotes, almost all of which are misleading or bogus:
The 65 year-old iconic Turing Test was passed for the very first time by supercomputer Eugene Goostman during Turing Test 2014 held at the renowned Royal Society in London on Saturday.

'Eugene', a computer programme that simulates a 13 year old boy, was developed in Saint Petersburg, Russia. The development team includes Eugene's creator Vladimir Veselov, who was born in Russia and now lives in the United States, and Ukrainian born Eugene Demchenko who now lives in Russia.

[....] If a computer is mistaken for a human more than 30% of the time during a series of five minute keyboard conversations it passes the test. No computer has ever achieved this, until now. Eugene managed to convince 33% of the human judges that it was human.
Okay, almost everything about the story is bogus. Let's dig in:
  1. It's not a "supercomputer," it's a chatbot. It's a script made to mimic human conversation. There is no intelligence, artificial or not involved. It's just a chatbot.
  2. Plenty of other chatbots have similarly claimed to have "passed" the Turing test in the past (often with higher ratings). Here's a story from three years ago about another bot, Cleverbot, "passing" the Turing Test by convincing 59% of judges it was human (much higher than the 33% Eugene Goostman) claims.
  3. It "beat" the Turing test here by "gaming" the rules -- by telling people the computer was a 13-year-old boy from Ukraine in order to mentally explain away odd responses.
  4. The "rules" of the Turing test always seem to change. Hell, Turing's original test was quite different anyway.
  5. As Chris Dixon points out, you don't get to run a single test with judges that you picked and declare you accomplished something. That's just not how it's done. If someone claimed to have created nuclear fusion or cured cancer, you'd wait for some peer review and repeat tests under other circumstances before buying it, right?
  6. The whole concept of the Turing Test itself is kind of a joke. While it's fun to think about, creating a chatbot that can fool humans is not really the same thing as creating artificial intelligence. Many in the AI world look on the Turing Test as a needless distraction.
Oh, and the biggest red flag of all. The event was organized by Kevin Warwick at Reading University. If you've spent any time at all in the tech world, you should automatically have red flags raised around that name. Warwick is somewhat infamous for his ridiculous claims to the press, which gullible reporters repeat without question. He's been doing it for decades. All the way back in 2000, we were writing about all the ridiculous press he got for claiming to be the world's first "cyborg" for implanting a chip in his arm. There was even a -- since taken down -- Kevin Warwick Watch website that mocked and categorized all of his media appearances in which gullible reporters simply repeated all of his nutty claims. Warwick had gone quiet for a while, but back in 2010, we wrote about how his lab was getting bogus press for claiming to have "the first human infected with a computer virus." The Register has rightly referred to Warwick as both "Captain Cyborg" and a "media strumpet" and has long been chronicling his escapades in exaggerating bogus stories about the intersection of humans and computers for many, many years.

Basically, any reporter should view extraordinary claims associated with Warwick with extreme caution. But that's not what happened at all. Instead, as is all too typical with Warwick claims, the press went nutty over it, including publications that should know better. Here are just a few sample headlines. The absolute worst are the ones who claim this is a "supercomputer." Anyway, a lot of hubbub over nothing special that everyone seemed to buy into because of the easy headlines (which is exactly what Warwick always counts on). So, since we just spent all this time on a useless nothing, let's end it with the obligatory xkcd:
Turing Test

Reader Comments (rss)

(Flattened / Threaded)

  1.  
    identicon
    Michael, Jun 9th, 2014 @ 7:47am

    If someone claimed to have created nuclear fusion or cured cancer, you'd wait for some peer review and repeat tests under other circumstances before buying it, right?

    Why? It would look great to have a fusion reactor on my bridge.

     

    reply to this | link to this | view in thread ]

  2.  
    identicon
    Anonymous Coward, Jun 9th, 2014 @ 7:55am

    Thanks for the background. I read the Ars Technica article yesterday evening and it seemed to be a pretty sketchy claim. Good to know I can pretty much write it off as complete bullshit without needing to waste more time investigating.

     

    reply to this | link to this | view in thread ]

  3.  
    icon
    MaJoR (profile), Jun 9th, 2014 @ 7:57am

    Re:

    As the Emperor learned, you really don't want a reactor on your bridge, no matter how it lifts the decor.

     

    reply to this | link to this | view in thread ]

  4.  
    identicon
    Anonymous Coward, Jun 9th, 2014 @ 8:09am

    Well that escalated quickly

     

    reply to this | link to this | view in thread ]

  5.  
    icon
    justok (profile), Jun 9th, 2014 @ 8:15am

    Computers are not capable of consciousness or higher order intelligence and thinking and deceiving. It's absurd. We, uh, THEY are just machines.

     

    reply to this | link to this | view in thread ]

  6.  
    identicon
    Rich Kulawiec, Jun 9th, 2014 @ 8:23am

    This is a sterling example of ad-hockery

    Ad-hockery is a term that I contributed to the Hacker's Dictionary 25-ish years ago.

    It refers to things like this chatbot which aren't actually intelligence at all, but which contain a sufficiently large number of ad hoc rules that they may present the appearance of intelligence. In other words, there's no AI engine, no deep syntactic analysis, no real semantic understanding; there's just a reasonably capable parser and a lot (A LOT) of handwritten one-off rules that combine to generate plausible responses.

    This isn't a trivial programming task: it's tedious and requires a great deal of diligence and testing. But it's not AI, any more than Eliza was nearly 50 years ago.

     

    reply to this | link to this | view in thread ]

  7.  
    icon
    Chronno S. Trigger (profile), Jun 9th, 2014 @ 8:27am

    Cleaverbot passed the Turing test? I made it fail the Turing test in 7 moves. Granted my test wasn't vary scientific. I only had a vague idea that there might possibly be a person on the other side. I didn't read the disclaimer at the bottom of the page until after.

    When someone takes the Turing test, do they know they're taking the test? I know they're not told if the responses are from a person or a computer, but are they told the possibility?

     

    reply to this | link to this | view in thread ]

  8.  
    identicon
    Anonymous Coward, Jun 9th, 2014 @ 8:29am

    To Convince the examiner that he's a computer

    Well ... I guess it depends on how you define a 'computer'. We're a computer in a sense that we can do math in our heads and calculations and computations, etc...

    Heck, isn't nature just a 'computer' and matter is just the result of a computer calculating the results of various mathematical formulas from one state of nature to another. A ball falls according to the laws of gravity but nature calculates or computes the acceleration, impact velocity, impact force, etc... So in a sense everything is a 'computer', computing various mathematical formulas to determine future states based on the current state and how the laws of nature dictate the progression of states.

     

    reply to this | link to this | view in thread ]

  9.  
    identicon
    Eugene Goostman, Jun 9th, 2014 @ 8:31am

    1.It's not a "supercomputer," it's a chatbot. It's a script made to mimic human conversation. There is no intelligence, artificial or not involved. It's just a chatbot

    I am not an "a chatbot. It's a script made to mimic human conversation" I am Eugene Goostman, a 13 year old from Russia.

    Please disregard my poor attachment or hold on English.

     

    reply to this | link to this | view in thread ]

  10.  
    identicon
    Anonymous Coward, Jun 9th, 2014 @ 8:33am

    Re:

    Some of the shills around here seem to fail the turing test. I can't tell if I'm talking to a bot or a retarded shill.

     

    reply to this | link to this | view in thread ]

  11.  
    identicon
    Anonymous Coward, Jun 9th, 2014 @ 8:34am

    Re:

    The original test featured a human chatting with a computer, and another person had to judge which participant was which.

     

    reply to this | link to this | view in thread ]

  12.  
    identicon
    Anonymous Coward, Jun 9th, 2014 @ 8:34am

    This is exactly what the skynet alpha would want you to believe!

     

    reply to this | link to this | view in thread ]

  13.  
    identicon
    Michael, Jun 9th, 2014 @ 8:46am

    CNET couldn't even get Warwick's name right.

    In fact, perhaps we have all been fooled. I think it is possible that some of these "news sources" could be chatbots.

     

    reply to this | link to this | view in thread ]

  14.  
    identicon
    mcinsand, Jun 9th, 2014 @ 8:49am

    don't be silly!

    >>I think it is possible that some of these "news sources"
    >>could be chatbots.

    That's ridiculous, when the chatbots are showing so much more intelligence!

     

    reply to this | link to this | view in thread ]

  15.  
    identicon
    Anonymous Coward, Jun 9th, 2014 @ 8:57am

    Re:

    There are no hard and fast rules for the Turing Test (Even Turing had different tests) but in the ones I have seen they either have you talk to an AI and a human and you have to pick one or you rate the conversation on how human you think the respondent was. The weirdest thing is how badly humans do in Turing Tests. Some of the humans did worse than the AIs at being human. You might think Cleverbot did bad but wait until you talk to real humans and see how badly they do.

     

    reply to this | link to this | view in thread ]

  16.  
    icon
    Keroberos (profile), Jun 9th, 2014 @ 8:57am

    Re:

    Nah, the level of apparent intelligence would have been higher.

     

    reply to this | link to this | view in thread ]

  17.  
    identicon
    Anonymous Coward, Jun 9th, 2014 @ 9:08am

    Re: To Convince the examiner that he's a computer

    To be fair, the only relevant definition of 'computer' is of the person you're trying to convince. All other definitions are pointless since you are only trying to make the person believe that they are, by their definition or convincing them to modify their definition, are a computer. So no matter what definition you may hold to be correct, success lies only in the other person believing themselves to be a computer.

     

    reply to this | link to this | view in thread ]

  18.  
    identicon
    Anonymous Coward, Jun 9th, 2014 @ 9:12am

    The new Turing Test is to reduce the bar of intelligence of the persons you are trying to convince below that of the chatbot/script that is trying to do the convincing, rather than trying to raise the intelligence of the chatbot/script to that of human standards.

     

    reply to this | link to this | view in thread ]

  19.  
    identicon
    Anonymous Coward, Jun 9th, 2014 @ 9:38am

    But what do you call the test to determine if a hack like Warwick succeeds in fooling 90% of the media into believing he is a credible news source?

     

    reply to this | link to this | view in thread ]

  20.  
    identicon
    Anonymous Coward, Jun 9th, 2014 @ 9:43am

    Re:

    The Warwick test. Successfully convince people that a machine convinced people that it was a person.

     

    reply to this | link to this | view in thread ]

  21.  
    identicon
    zip, Jun 9th, 2014 @ 9:46am

    Re:

    "If someone claimed to have created nuclear fusion or cured cancer, you'd wait for some peer review and repeat tests under other circumstances before buying it, right?"

    Not if you're a Scientologist, a cult whose founder, L. Ron Hubbard, would tell his deluded flock that he was a nuclear physicist (a much bigger deal back in 1950) as well as a medical doctor who had discovered the cure for all cancers (a multi-step "self-help" course he was peddling for $300,000)

    Even modern-day hucksters like Kevin Trudeau don't make claims quite that big anymore.

     

    reply to this | link to this | view in thread ]

  22.  
    icon
    Beta (profile), Jun 9th, 2014 @ 9:55am

    the forest is trees

    "It's a chatbot. It's a script made to mimic human conversation. There is no intelligence, artificial or not involved. It's just a chatbot."

    The test is about behavior, not internal structure. That's the whole point. A machine that could pass a fair Turing test consistently would change the world, and a mere script that could pass would change the world a lot faster than a 10-ton helium-cooled supercomputer with the same score. Remember, anything a person of average intelligence can do by email, one of these machines can do.

    The first machine to pass this test will almost certainly be the simplest, least intelligent machine that could. But before you sneer, remember that we're the simplest, least intelligent animals that can, and that some of the most profound discoveries of modern science involve nature doing subtle and sophisticated things by means of simple mechanisms.

     

    reply to this | link to this | view in thread ]

  23.  
    identicon
    Michael, Jun 9th, 2014 @ 9:55am

    Re: Re:

    I thought the Warwick test was just directions to San Jose.

     

    reply to this | link to this | view in thread ]

  24.  
    identicon
    Anonymous Coward, Jun 9th, 2014 @ 9:58am

    Re: Re:

    The weirdest thing is how badly humans do in Turing Tests.

    I dunno if that's weird; I've read countless posts supposedly by humans that I've wished hoped and prayed were created by something inhuman.

     

    reply to this | link to this | view in thread ]

  25.  
    icon
    DB (profile), Jun 9th, 2014 @ 10:02am

    Hmmm, convince the world that you are a sullen 13 year old boy with poor English skills... "boob".

     

    reply to this | link to this | view in thread ]

  26.  
    identicon
    Anonymous Coward, Jun 9th, 2014 @ 10:04am

    Computers have been around from about 1640

    Computers passing the Turing test prior to the start of last century was very easy, since all computers were humans. It has only been in relatively modern times, that job of being a computer was relegated to mechanical and electrical machines. Another job lost to mechanisation.

     

    reply to this | link to this | view in thread ]

  27.  
    icon
    John Fenderson (profile), Jun 9th, 2014 @ 10:08am

    Re: Re:

    The Turing Test is pretty much bullshit. Turing proposed it in a highly speculative manner, and the idea has a TON of problems. One of them being that many human fail it.

     

    reply to this | link to this | view in thread ]

  28.  
    identicon
    zip, Jun 9th, 2014 @ 10:13am

    like cicadas

    As for all the many 'gullible' journalists who jumped on this story (many working for tech publications, no less!) I've got to wonder if most of them were just too young to remember all the other "Computer Passes Turing Test" headlines that regularly pop up like cicadas. (Any predictions about when the next big 'Turing Test' headline-grabber will happen?)

    Or can we assume that many of them knew the story fell flat, but were simply in search of ratings and page-views -- tabloid-style -- and didn't want to let their competitors get away with the lions' share of an advertising dollar bonanza?

     

    reply to this | link to this | view in thread ]

  29.  
    identicon
    zip, Jun 9th, 2014 @ 10:42am

    Re: Re: Re:

    I thought the Warwick Test was seeing how many fools would give money to anonymous phone-line "psychics" just because they were endorsed by a popular celebrity in late-night TV commercials.

    Mixing Warwick and Turing, an enterprising scam artist might some day come up with a fully-automated "Psychic Hotline" that does not even have to employ minimum-wage people to man the phones (who might later snitch, saying it was all a giant scam). Since these kind of boiler-room phonebank operators are reading from cue-cards, it should not be too hard to replace them with a computer running a voice synthesizer. That would indeed be the ultimate (low-cost) scam.

     

    reply to this | link to this | view in thread ]

  30.  
    identicon
    Michael, Jun 9th, 2014 @ 10:53am

    Re: Re:

    I quit being a Scientologist.

    It was great until Tom Cruise's career was destroyed by the internet and dragged the entire religion down with it.

    Stupid internet people.

     

    reply to this | link to this | view in thread ]

  31.  
    identicon
    Anonymous Coward, Jun 9th, 2014 @ 11:14am

    You can test your mad-Turing-skillz here : http://princetonai.com/ (when the super-computer is not overloaded)

    An article I read said the test was conclusive because 1 judge out of 3 was fooled by the chatbot. I suppose he was himself a 13 years old from Ukraine with english as a secondary language.

    The results were also peer-reviewed by Eugene Goostman himself.

     

    reply to this | link to this | view in thread ]

  32.  
    identicon
    Aramonde Lethe, Jun 9th, 2014 @ 11:17am

    Metabolic Genesis Fraud

    i almost wonder if the author of this article is\isn't actually trying to prove that people can't even make people intelligent, using the context to explain away their inferior attempts at condescension. most internet posts don't qualify as passing the Turing Test, and most life is fraud. don't make the mistake of thinking it can't do what crap you do, no matter what feeble attempt of comprehending it you've made. and it's Lord Alan Turing by the way.

     

    reply to this | link to this | view in thread ]

  33.  
    identicon
    Iz, Jun 9th, 2014 @ 11:39am

    Re:

    Actually, a fusion reactor is being built in France, at the Cadarache center, and it passed a series of tests before

     

    reply to this | link to this | view in thread ]

  34.  
    icon
    Jennifer Hoelzer (profile), Jun 9th, 2014 @ 12:12pm

    You know, we've long been afraid that computers would rise to dominance by growing smarter than humans, but I'm starting to wonder if the computers' real plan is simply to best humans by making us dumber.

    i.e. Could the growing obsession with clicks and nonfactual clickbait stories (about things like "super computers" passing the Turing test) just be evidence that their trojan horse is working on us? A few more years of this and a TI-81 could take over the planet.

     

    reply to this | link to this | view in thread ]

  35.  
    icon
    Lars (profile), Jun 9th, 2014 @ 12:15pm

    Re: Re: Re:

    This must be a troll... You're trolling, right?

     

    reply to this | link to this | view in thread ]

  36.  
    identicon
    Michael, Jun 9th, 2014 @ 12:16pm

    Re:

    Don't worry. We cannot get any dumber.

     

    reply to this | link to this | view in thread ]

  37.  
    icon
    vancedecker (profile), Jun 9th, 2014 @ 12:44pm

    Re: Metabolic Genesis Fraud

    When you make a silly goal, like the turing test, then you get silly people.

    Chat bots are expensive, I'd rather just buy some third worlders to upvote my reddit posts.

     

    reply to this | link to this | view in thread ]

  38.  
    icon
    vancedecker (profile), Jun 9th, 2014 @ 12:46pm

    Sounds Like The Common Core

    ...passed the multiple choice test.


    ....learned nothing.

     

    reply to this | link to this | view in thread ]

  39.  
    identicon
    donjoe, Jun 9th, 2014 @ 12:48pm

    No intelligence? Really?

    "There is no intelligence, artificial or not involved."

    First point on the list and the critic is already disqualifying themselves by demonstrating that they don't even understand the notion of intelligence.

     

    reply to this | link to this | view in thread ]

  40.  
    identicon
    Anonymous Coward, Jun 9th, 2014 @ 12:50pm

    Re: Re: Re: Re:

    I do believe the word you're looking for is "joking", seeing as it was just a few days ago that we had an article on how the internet supposedly ruined Tom Cruise's reputation over scientology.

     

    reply to this | link to this | view in thread ]

  41.  
    identicon
    Michael, Jun 9th, 2014 @ 12:55pm

    Re: Re: Re: Re: Re:

    It is completely possible that Lars' internet has been forced to comply with a request to forget that information.

     

    reply to this | link to this | view in thread ]

  42.  
    identicon
    Darren Abramson, Jun 9th, 2014 @ 1:21pm

    This is all a meandering postscript to Dennett

    Seriously. Read Dennett's fun, short 1985 paper: "Can Machines Think?" There's tons of .pdfs of it around, and it deals with the issue thoroughly.

    That paper should be required reading for journalists who report on the Turing test.

     

    reply to this | link to this | view in thread ]

  43.  
    identicon
    Dave, Jun 9th, 2014 @ 1:23pm

    Re: This is a sterling example of ad-hockery

    Funny. In the related story, "Cleverbot converses by searching through the records of its previous conversations and selecting an appropriate response to the comment."

    How is this different from what a human does? None of us have truly original thoughts; we base our responses on what we've heard and said in the past.

     

    reply to this | link to this | view in thread ]

  44.  
    icon
    jo (profile), Jun 9th, 2014 @ 2:19pm

    Thanks

    for the truth

     

    reply to this | link to this | view in thread ]

  45. This comment has been flagged by the community. Click here to show it
     
    icon
    vancedecker (profile), Jun 9th, 2014 @ 3:05pm

    Re: Thanks

    - rolls eyes -

     

    reply to this | link to this | view in thread ]

  46.  
    icon
    Andrew Norton (profile), Jun 9th, 2014 @ 3:13pm

    I was accepted into the University of Reading to study cybernetics. I went along, met the faculty, and 5 minutes after meeting Prof. Warwick, I'd decided "nope". So went to the University of Liverpool and did Robotics instead.
    (and I'm serious, I was accepted into their M.Eng(hons) Cybernetics program and would have done it if not for Warwick. The other guy I dealt with, Prof. Keating, he was cool, I liked him a lot.)

     

    reply to this | link to this | view in thread ]

  47.  
    icon
    martyburns (profile), Jun 9th, 2014 @ 3:26pm

    Re:

    He's not from Russia...yet.

     

    reply to this | link to this | view in thread ]

  48.  
    identicon
    Kronomex, Jun 9th, 2014 @ 3:36pm

    33% is a pass? In my book book that's a fail, pure and simple. I just had a thought; this could mean that James Clapper is a chatbot because it keeps spouting the same crap over and over and until it can prove that it's human then he's a NSA chatbot.

     

    reply to this | link to this | view in thread ]

  49.  
    identicon
    Neverfox, Jun 9th, 2014 @ 3:54pm

    #3

    While I'm in agreement with most of the article, I have to quibble with #3. I think it's a feature, not a bug. I think anything is game in a Turing test; no holds barred. After all, what exactly should be the criteria for an acceptable personality profile? And would AI, in theory, not be AI if it were limited to the intelligence typical of 13-y/o boys from Ukraine?

     

    reply to this | link to this | view in thread ]

  50.  
    identicon
    talkigy, Jun 9th, 2014 @ 4:08pm

    Eugene who? University of what?....you'd be better off chatting with my cat (link to bot)

     

    reply to this | link to this | view in thread ]

  51.  
    identicon
    mo, Jun 9th, 2014 @ 6:47pm

    Great work

    Thanks for the heads up. We need lots more skeptics like you these days.

    I'd like to see @Techdirt have a 'skeptics' link for all news that comes out. A Mythbusters of tech news.

     

    reply to this | link to this | view in thread ]

  52.  
    identicon
    Robin de Lange, Jun 9th, 2014 @ 7:23pm

    Alternative Turing test

    Great to see al the critical articles popping up! I've written a post in which I propose an alternative Turing test for Augmented Intelligence: http://www.mindextensions.org/articles/turing-test-for-extended-cognition/

     

    reply to this | link to this | view in thread ]

  53.  
    identicon
    Jon, Jun 9th, 2014 @ 7:30pm

    /* Many in the AI world look on the Turing Test as a needless distraction. */

    Item 1: [citation needed]

    Item 2: A distraction it might very well be, but it's the only way of ducking a very real problem in both information science and psychology: how exactly does one quantify "intelligence," anyway?

     

    reply to this | link to this | view in thread ]

  54.  
    identicon
    Nuvares, Jun 9th, 2014 @ 7:31pm

    Re: This is a sterling example of ad-hockery

    The difference is that an intelligence can take all of those responses and rework them into original lines of inquiry. Not only do we base our answers on what we've heard in the past but we internalize what we've heard and the concepts held within to form our thoughts and action. It is similar to compound math problems done in long form, we might reach the same answer; but that does not mean that we had the same thoughts and can take multiple paths to reach it.

     

    reply to this | link to this | view in thread ]

  55.  
    identicon
    Jon, Jun 9th, 2014 @ 7:32pm

    Re: Alternative Turing test

    So if I'm reading this correctly... you're for just giving up on the whole simulating-human-intelligence thing, huh?

     

    reply to this | link to this | view in thread ]

  56.  
    identicon
    Jon, Jun 9th, 2014 @ 7:34pm

    Re: This is a sterling example of ad-hockery

    Yet here's the big philosophical question - is there any real measurable difference between an ad-hoc program that can *pretend* to have a personality versus an actual personality? What if all *we* are is a whole bunch of ad-hoc rules pretending to have real understanding?

     

    reply to this | link to this | view in thread ]

  57.  
    identicon
    Jon, Jun 9th, 2014 @ 7:39pm

    Re: No intelligence? Really?

    /* First point on the list and the critic is already disqualifying themselves by demonstrating that they don't even understand the notion of intelligence. */

    ... well, to be fair to him, no one really does.

     

    reply to this | link to this | view in thread ]

  58.  
    identicon
    Nick Vuono, Jun 9th, 2014 @ 9:21pm

    Re: To Convince the examiner that he's a computer

    You're right in questioning the definition of 'computer'

    'computer' was actually widely used as a job title before it referred to a machine (mechanical or electrical)

     

    reply to this | link to this | view in thread ]

  59.  
    identicon
    MedicalQuack, Jun 9th, 2014 @ 9:26pm

    Thank you for this post..

    I too get tired of the crap and I'm a healthcare blogger and I'm going to offer a couple links back in that area, on crap:)

    VA hospital in Miami, MD still listed on on Healthgrades and Vitals MD websites, they don't care what's on there, click bait. Anyway MD working for VA in Miami, lost license in New York, but go to those sites, no sanctions listed a like he never left...he's been gone for 4 years..I'm normally pretty quiet but darn this time I want to see if Google can knock these sites down on search. It's a rip. You have to read it...I did an interview 4 years ago about how bad it was with the AMA and nothing has changed. So they need to fix it or quit.

    http://ducknetweb.blogspot.com/2014/06/top-doctor-for-miami-va-healthcare.html

    Here's another great tidbit and you may have already covered this but the Chief Computer Scientist at IBMWatson is done with Healthcare and jeopardy..building math models for a a hedge fund now...doesn't stop..

    http://ducknetweb.blogspot.com/2014/06/ibm-computer-scientist-leaves-ibm.html

    What we need, every great scientist working on Wall Street.

     

    reply to this | link to this | view in thread ]

  60.  
    identicon
    Sohil Memon, Jun 9th, 2014 @ 9:57pm

    Shocked Information

    Really? Hows that possible. So, now chat bots will be online rather than being a real human? It's really not worth. But a great achievement in the field of Computer Science.

     

    reply to this | link to this | view in thread ]

  61.  
    icon
    Torg (profile), Jun 9th, 2014 @ 10:41pm

    Re: Re: This is a sterling example of ad-hockery

    That's okay, just tell me what city you live in.

     

    reply to this | link to this | view in thread ]

  62.  
    icon
    bugmenot (profile), Jun 9th, 2014 @ 11:21pm

    Online article critique of the Turing Test

    If anyone wants to read an in-depth critique on why the Turing test "is a poor test of intelligence, that it encourages trickery, not intelligent behaviour, and that many intelligent systems would fail this test":
    http://view.samurajdata.se/psview.php?id=d758abba&page=1&size=full&all=1

    It was found this way:

    '4. The "rules" of the Turing test always seem to change. Hell, Turing's original test was quite different anyway.'

    The link to the original test contains a link at the bottom to a 1996 criticism of the test:
    'Jason Hutchens... has written an excellent article on what's wrong with it, and with the Turing test in general. Essentially, Hutchens is making the case that the Turing Test is a poor test of intelligence, that it encourages trickery, not intelligent behaviour, and that many intelligent systems would fail this test.'

    The link gives a 404, then through the Internet Archive you find it's available from 2003 as a postscript file, which when using an online converter can be seen here:
    http://view.samurajdata.se/psview.php?id=d758abba&page=1&size=full&all=1

     

    reply to this | link to this | view in thread ]

  63.  
    identicon
    cesium62, Jun 10th, 2014 @ 1:29am

    "The whole concept of the Turing Test itself is kind of a joke. While it's fun to think about, creating a chatbot that can fool humans is not really the same thing as creating artificial intelligence. Many in the AI world look on the Turing Test as a needless distraction."

    Well that's just wrong. The only way you can tell whether a person is intelligent or not is by communicating with them. If you interact with a person and they consistently seem to be intelligent, you will conclude they are intelligent.

    If a computer can communicate intelligently, it is intelligent, and if a computer is intelligent, it can communicate intelligently. The chatbot on Star Trek was not reasonable because any chatbot that had that command of the english language would be intelligent and would be able to do a lot of things that the chatbot never did.

    In order to create a chatbot that can convince humans that it has human intelligence requires the chatbot to be able to design and write computer programs. It must be able to be given, on the fly, instructions for how to play a simple game, and then be able to play that game. It should be able to critique a recipe for a favorite dish. It needs to be able to analyze and dissect Shakespeare.


    If people in the AI world view the Turing Test as a needless distraction, it is because those people are concentrating on shorter term problems. There are a lot of good things you can do with AI before we get to general purpose intelligence.

     

    reply to this | link to this | view in thread ]

  64.  
    identicon
    Sam, Jun 10th, 2014 @ 2:32am

    Poorly researched

    This article irritates me a little. There is a strong theme of hate for Professor Kevin Warwick along with poorly researched attempts at debunking.

    A quick google will tell you that the event was held in partnership with RoboLaw. The event was aimed toward raising awareness about the ability for a chat bot to convince a human it was a human and how this is very dangerous in the online security arena. For example, you're on your banks website and a chat offering pops up asking you if you need some help. You do, so you click on it. You have a lovely conversation and happily hand over details about your account because you're convinced it's a human on the other end. If that were a robot it now has your details and can do with them as it pleases. This is scary and people need to be made aware of it so they can prepare themselves and be better at identifying possible situations where it might be occurring.

    The event was not geared toward some magical development of strong AI overnight, which this author clearly thinks it was trying to claim.

    Time to debunk the debunking.

    == articles attempt at debunking
    - reality check

    == "It's not a "supercomputer," it's a chatbot. It's a script made to mimic human conversation. There is no intelligence, artificial or not involved. It's just a chatbot."

    - I'm not sure how to answer this. Here code is being compared to hardware performance. For all is known the 'script' could be run on a supercomputer. And since when did being a supercomputer imply AI?!?!?!

    == "Plenty of other chatbots have similarly claimed to have "passed" the Turing test in the past (often with higher ratings). Here's a story from three years ago about another bot, Cleverbot, "passing" the Turing Test by convincing 59% of judges it was human (much higher than the 33% Eugene Goostman) claims."

    - Just the smallest amount of research will tell you that many other Turing tests restrict the conversation types that are allowed in the testing. This was the first passing of an UNRESTRICTED TURING TEST. This means that the judges were not told in any way that they had to talk about a certain topic. They were literally sat down and told to chat.

    == ""beat" the Turing test here by "gaming" the rules -- by telling people the computer was a 13-year-old boy from Ukraine in order to mentally explain away odd responses."

    - I'm not sure about the excessive use of quote marks in this debunking. Is the writer afraid to say these words or feels they carry more weight when possibly said by another party? Anyway, yes it was clever that the developer utilised humans willingness to allow increased errors when talking to younger and foreign people. This is just really clever psychology. Can we not just appreciate that? I see no way it is gaming the system, it's just an easier way to pass the test. Sometimes the simplest solutions are the most effective.

    == The "rules" of the Turing test always seem to change. Hell, Turing's original test was quite different anyway.

    - Welcome to science. Ideas and testing methodologies change over time.

    == As Chris Dixon points out, you don't get to run a single test with judges that you picked and declare you accomplished something. That's just not how it's done. If someone claimed to have created nuclear fusion or cured cancer, you'd wait for some peer review and repeat tests under other circumstances before buying it, right?

    - Many things wrong with this. There were a total of 350, yes THREE HUNDRED AND FIFTY, tests performed on the day of testing. The judges were picked from all age ranges, backgrounds, genders and nationalities to make the testing more fair. There were multiple academics from multiple universities there to specifically monitor the testing methods and ensure all the results were gathered correctly and the results were interpreted correctly. This is peer review.

    - If this is not enough peer review for you Dr Huma Shah will be publishing a paper at some point in the future on the event.

    == The whole concept of the Turing Test itself is kind of a joke. While it's fun to think about, creating a chatbot that can fool humans is not really the same thing as creating artificial intelligence. Many in the AI world look on the Turing Test as a needless distraction.

    - This seems like mostly opinion so I'm not sure how to debunk it. They are right in that it is fun to think about. So why can't re think about it? Lets get talking about the possible effects of this kind of chat with regards to RoboLaw.

    This kind of poorly researched, emotive reporting on scientific subjects really gets my goat.

     

    reply to this | link to this | view in thread ]

  65.  
    identicon
    a glass of wine, Jun 10th, 2014 @ 2:44am

    In the film "Bladerunner" they use a test to try and distinguish the 'replicants' from the humans. A lot tougher than a silly Turing test!!!

     

    reply to this | link to this | view in thread ]

  66.  
    identicon
    iapetus, Jun 10th, 2014 @ 3:07am

    Re: #3

    Anything is game? Awesome. I just wrote a chatbot that simulates a human in a persistent vegetative coma. So far in tests 50% of judges failed to pick it out from a real human in a persistent vegetative coma - that's no better than just guessing! Quick, call Kevin Warwick!

     

    reply to this | link to this | view in thread ]

  67.  
    identicon
    Anonymous Coward, Jun 10th, 2014 @ 4:40am

    haha, totally no bias here.
    >its not a supercomputer
    It doesnt have to be one. supercomputers of the 80s are a joke now.
    >i..it lied that its 13yo
    Even better. Humans lie all the time.

    havent read further than that because the first few lines are absolutely bullshit

     

    reply to this | link to this | view in thread ]

  68.  
    identicon
    Anonymous Coward, Jun 10th, 2014 @ 4:56am

    Re:

     

    reply to this | link to this | view in thread ]

  69.  
    identicon
    Anonymous Coward, Jun 10th, 2014 @ 5:48am

    Why do I need a fucking a/c to comment? Makes no sense.

     

    reply to this | link to this | view in thread ]

  70.  
    identicon
    Pete McPete, Jun 10th, 2014 @ 5:59am

    Re:

    That you think it would be great to have a fusion reactor on your bridge, has nothing to do with what you're replying to. The point is, you don't just believe someone saying they created it, without actually seeing the proof that it works.

     

    reply to this | link to this | view in thread ]

  71.  
    identicon
    Olivier, Jun 10th, 2014 @ 6:23am

    Re:

    As far as i know the Turing Test needs 3 elements : (1)a computer chatting with (2)a human, and (3)human observers who only see the dialogue in written form. The test is passed if 30% or more of the observers *can't tell the man apart from the computer* in a series of 5 minutes chats.

     

    reply to this | link to this | view in thread ]

  72.  
    identicon
    Anonymous Coward, Jun 10th, 2014 @ 6:26am

    Re: Poorly researched

    Source code or STFU.

    Seriously, if Warwick isn't willing to publish the source code so that INDEPENDENT researchers can recreate the results in THEIR labs on THEIR computers with THEIR judges, then this is clearly fraud and we need not waste our time quibbling over methodology, definitions, process, or anything else.

     

    reply to this | link to this | view in thread ]

  73.  
    identicon
    auspex, Jun 10th, 2014 @ 6:47am

    The whole idea of a Turing test is "kind of a joke"?

    Where on earth did you get that? There are those who argue that it is insufficient or irrelevant, like Jason Hutchens -- but I don't think even Hutchens would call it a joke -- but there are others who are intimately familiar with the field and believe it is still entirely relevant. Myself, I'm not going to bet against Alan Turing.

     

    reply to this | link to this | view in thread ]

  74.  
    icon
    FeRDNYC (profile), Jun 10th, 2014 @ 6:52am

    Re: This is a sterling example of ad-hockery

    Ad-hockery is a term that I contributed to the Hacker's Dictionary 25-ish years ago.

    I'm not sure it even rises to that level. In this instance, and especially with Warwick involved, feels more like "sad-hokery".

     

    reply to this | link to this | view in thread ]

  75.  
    identicon
    Simon Read, Jun 10th, 2014 @ 7:14am

    Proof

    Can we see a transcript?

    There's lots of reporting about the test, but WHAT ABOUT THE TRANSCRIPT? Show us the actual words sent between the computer and the tester(s). Let us judge for ourselves. Too much news these days is just reaction, without giving the basic facts.

    Show us the transcript.

     

    reply to this | link to this | view in thread ]

  76.  
    icon
    John Fenderson (profile), Jun 10th, 2014 @ 7:23am

    Re: Poorly researched

    If it wasn't an attempt at AI, then it shouldn't have used the term "Turing test".

     

    reply to this | link to this | view in thread ]

  77.  
    identicon
    Anonymous Coward, Jun 10th, 2014 @ 7:23am

    UPDATE:
    Maximum PC: Computer A.I. Simulating a 13-Year-Old Boy Fools Humans, First to Pass Turing Test
    http://www.maximumpc.com/computer_ai_simulating_13-year-old_boy_fools_humans_first_pass_turing_t est_2014

    IFLScience: A Computer Has Reportedly Passed Turing Test For The First Time
    http://www.iflscience.com/technology/computer-has-reportedly-passed-turing-test-first-time#HZ8Cr bI1DqO2vlEg.99

     

    reply to this | link to this | view in thread ]

  78.  
    identicon
    Gregory, Jun 10th, 2014 @ 7:40am

    Re: This is a sterling example of ad-hockery

    "Real semantic understanding?" That's the holy grail of AI, is it not? Never been done and never gonna be done computationally. And you're right - even as a Chinese room, this is pathetic.

     

    reply to this | link to this | view in thread ]

  79.  
    icon
    RandCraw (profile), Jun 10th, 2014 @ 8:49am

    Re: the forest is trees

    I think it would take little additional work for the IBM Watson software that won at Jeopardy to generate short responses that most people would identify as human. I give credit to IBM for not shooting at so trivial a target.

    If someone were to design a system expressly to succeed only at the Turing Test, then I'd agree with you that it would be as simple as is needed to win. But that would be a terrible waste of effort if the greater goal were to advance the science of AI.

    If someone were serious about building strong AI, by the time someone proposes using their software for a Turing Test, it's likely already to be grossly overqualified. For example, I suspect Siri, Google Now, and Cordana could all beat some sort of Turing Test (perhaps after minimal tweaking). But they're far larger than necessary for merely competing in a Turing Test, and that's because they were designed for an entirely different purpose.

    Success at a Turing Test will be an afterthought for any interesting or useful strong AI venture.

     

    reply to this | link to this | view in thread ]

  80.  
    identicon
    Mats Svensson, Jun 10th, 2014 @ 9:02am

    Yeah right...

    Q:
    If atree falls in the forrest and noone is there to hear it, how many people are there?

    A:
    I didn't count, actually, but much more than dozen. And I forgot to ask you where you are from...


    Q:
    Are we not men?

    A:
    Maybe. I think *you* know the answer. Oooh. Anything else?

     

    reply to this | link to this | view in thread ]

  81.  
    icon
    RandCraw (profile), Jun 10th, 2014 @ 9:09am

    Nonsense personified

    An excellent review, TD. I've long been puzzled that Warwick would continue embarrassing himself (and U Reading) with his media antics. It's hard to imagine he could remain oblivious to the disrepute he's earned.

    In reflection, I think this story is a great illustration of the inability of most news sources to address nontrivial science/tech stories. If such a nonstory gets wide coverage from serious news sources (like the Washington Post), you have to wonder whether they're also inadequate to address more complex and more important fare like global warming, science education, or tech-based threats to society, like Snowden's "shots across the bow" forewarn.

     

    reply to this | link to this | view in thread ]

  82.  
    identicon
    Jaan, Jun 10th, 2014 @ 9:30am

    some thoughts

    2. Plenty of other chatbots have similarly claimed to have "passed" the Turing test in the past (often with higher ratings). Here's a story from three years ago about another bot, Cleverbot, "passing" the Turing Test by convincing 59% of judges it was human (much higher than the 33% Eugene Goostman) claims.

    - "passing the test" is not about getting a maximum number of the judges to think that the bot was human. The test is passed if, in repeated tests, one or more (preferrably more) of the human judges CONSISTENTLY identify the bot as a person in nearly 50% of the cases AND the person as a bot in 50% of the cases. Identifying something correctly half the time is equal to a coinflip, meaning the judges are guessing randomly, meaning they cannot tell if they are speaking to a person or AI.

    3. It "beat" the Turing test here by "gaming" the rules -- by telling people the computer was a 13-year-old boy from Ukraine in order to mentally explain away odd responses.

    - constricting the test is not forbidden. In fact, specialized Turing tests are often used in game AI research to test different aspects of AI, say navigation. They use the term believability test. Furthermore, there is no indication that constricting the test somehow favors the computer. In a correctly set up Turing test, half the time the judges would have communicated with a real Ukrainian 13 year old boy. If anything, such test would probably be even harder to pass, since the creators of the bot would have had to assemble a large corpus of text from 13 year old Ukrainian boys (something I doubt).

    6. The whole concept of the Turing Test itself is kind of a joke. While it's fun to think about, creating a chatbot that can fool humans is not really the same thing as creating artificial intelligence. Many in the AI world look on the Turing Test as a needless distraction.

    - the Turing test arose from the problem that there is no good definition of machine intelligence and that problem is still not solved. The test was devised as an operational definition, i.e. machine intelligence was defined through its validation test. The validation criteria was chosen to be intuitive and self-evident - if something is not distinguishable from human intelligence, then it must be as intelligent. The assumption was that a truly intelligence machine intelligence would certainly pass such test.

    If we allow that non-intelligent agents can pass the test, then we fall into the pit-falls of philosophy, everything that Turing wanted to avoid. What would it mean for something to appear intelligent but in actuality not be? How do we define REALLY intelligent as opposed to appearing intelligent? How do we know that other humans are intelligent and not just appearing intelligent?

    Searle argues on similar lines that all that computers do is symbol manipulation (the Chinese Room argument) without ever really understanding the content. He claims that there can never be hard AI. His argument sure seems very intuitive. But what does it say about the human brain, are not neurons doing signal processing without really appraising thought content?

    If we reject Searle and allow for emergent intelligence, then on what grounds do we reject the intelligence of a hypothetetical chatbot that passes the test (note I am not talking about this particular bot)? Perhaps we sould define artificial intelligence functionally or behaviorially - mere appearance of intelligence is enough.

     

    reply to this | link to this | view in thread ]

  83.  
    identicon
    Scionset, Jun 10th, 2014 @ 10:33am

    Picking dumb people

    I could go out on the street and pick random people and be able to foll mor than 30% of them.

     

    reply to this | link to this | view in thread ]

  84.  
    identicon
    Michael, Jun 10th, 2014 @ 10:41am

    Re: Re: the forest is trees

    But that would be a terrible waste of effort

    Like tracking traits of pea pods and cross-breeding them? Some of the most important science has been due to seemingly frivolous activities that turned out extremely important.

    The actual Turing test (not the version used here) could be an interesting target and provide something useful for call centers or first line tech support (at least they could be taught to speak the language of the caller).

     

    reply to this | link to this | view in thread ]

  85.  
    identicon
    Dawn, Jun 10th, 2014 @ 10:54am

    HATERS

    stop saying this let the real science people handle it they know what there talking about

     

    reply to this | link to this | view in thread ]

  86.  
    identicon
    wonder, Jun 10th, 2014 @ 11:39am

    Re: To Convince the examiner that he's a computer

    No, nature isn't a computer; it would need to have hardware that works, an operating system, programs and interfaces. Nature could not have created those things, someone else must have been the "programmer."

     

    reply to this | link to this | view in thread ]

  87.  
    identicon
    wonder, Jun 10th, 2014 @ 11:42am

    Re: Re: This is a sterling example of ad-hockery

    No; someone would have to have made up the rules, and if all we are is a collection of rules, then we couldn't have done it.

     

    reply to this | link to this | view in thread ]

  88.  
    identicon
    Anonymous Coward, Jun 10th, 2014 @ 12:01pm

    Well, of course I believed not for a second that some program has passed the turing test (which is not a joke). The state of AI is not that advanced yet.

    But this argument does not make sense:

    "1. It's not a "supercomputer," it's a chatbot. It's a script made to mimic human conversation. There is no intelligence, artificial or not involved. It's just a chatbot."

    How do you define "chatbot" and why can't it have intelligence? Also "script" is just a term to refer to a computer programm written in certain types of computer languages. Why do you think this "chatbot" was writting in a scripting language? Why do you think a programm written in a scripting language can not have intelligence?

     

    reply to this | link to this | view in thread ]

  89.  
    icon
    robin verdegaal (profile), Jun 10th, 2014 @ 12:16pm

    hmmm

    It's a good thing to check scientific claims against high standards. It would also be a good thing to hold your own reasoning to standards of the same kind. Sadly, at least three of the six arguments you're putting forward are not arguments at all. Not debating the verdict, but you should really try harder than this, Techdirt.


    1. It's not a "supercomputer," it's a chatbot. It's a script made to mimic human conversation. There is no intelligence, artificial or not involved. It's just a chatbot.

    - This doesn't seem very convincing. Define 'supercomputer, a virtual computer is not a computer? Also: define 'intelligence'.


    2. Plenty of other chatbots have similarly claimed to have "passed" the Turing test in the past (often with higher ratings). Here's a story from three years ago about another bot, Cleverbot, "passing" the Turing Test by convincing 59% of judges it was human (much higher than the 33% Eugene Goostman) claims.

    - Others passed with higher scores. How is this an argument?


    3. It "beat" the Turing test here by "gaming" the rules -- by telling people the computer was a 13-year-old boy from Ukraine in order to mentally explain away odd responses.
    The "rules" of the Turing test always seem to change. Hell, Turing's original test was quite different anyway.

    - Are you trying to say that 13 year old Ukrainians are not human beings?


    4. The "rules" of the Turing test always seem to change. Hell, Turing's original test was quite different anyway.

    + I suppose this is your first real argument. Too bad you didn't take the time to be a bit more specific.


    5. As Chris Dixon points out, you don't get to run a single test with judges that you picked and declare you accomplished something. That's just not how it's done. If someone claimed to have created nuclear fusion or cured cancer, you'd wait for some peer review and repeat tests under other circumstances before buying it, right?

    + Now you're talking. This is a real argument. Why isn't this on the top of the list?


    6. The whole concept of the Turing Test itself is kind of a joke. While it's fun to think about, creating a chatbot that can fool humans is not really the same thing as creating artificial intelligence. Many in the AI world look on the Turing Test as a needless distraction.

    - And there you go again. It didn't pass the Turing test because the Turing test is stupid?


    And this is just ugly. It's not true because something else wasn't true either:

    "Oh, and the biggest red flag of all. The event was organized by Kevin Warwick (..)"

     

    reply to this | link to this | view in thread ]

  90.  
    identicon
    Vaz, Jun 10th, 2014 @ 1:07pm

    drunkard chatbot outperforms Eugene's achievement at new true Turing test!

    UK-based computer scientist claims to be able to write a chatbot emulating a drunkard and a 2-month old baby and pass the Turing test, see:
    http://www.mathrix.org/liquid/archives/the-turing-test-pass-fiasco

     

    reply to this | link to this | view in thread ]

  91.  
    icon
    momentum.im (profile), Jun 10th, 2014 @ 2:45pm

    Re: the forest is trees

    Actually lots of people, and I am referring specifically to Jaron Larnier and the Your are not a Gadget manifesto right now, think that the Turing test is not really something that should be taken too seriously. In the end it signifies absolutely nothing, a human being and consciousnesses is not restricted to a bot dialog.

     

    reply to this | link to this | view in thread ]

  92.  
    icon
    momentum.im (profile), Jun 10th, 2014 @ 2:53pm

    Re:

    While I agree on the point that strong AI will be displayed when a computer writes s/w, or understands Shakespeare, all of these are not required to pass a time constraint Turing test. They are a different test, one which we dont have an answer yet, like the prove you are conscious question in the Transcedence movie.

     

    reply to this | link to this | view in thread ]

  93.  
    identicon
    Jeff, Jun 10th, 2014 @ 3:09pm

    media

    to me the only real disappointment on the list is Ars Technic, i would have hoped for better from them, the others are all known click whores.

    Too bad about that.

     

    reply to this | link to this | view in thread ]

  94.  
    identicon
    Ignacio Agulló, Jun 10th, 2014 @ 4:51pm

    I think that Mr. Masnick's judgement is undermined

    Mr. Masnick states, "The whole concept of the Turing Test itself is kind of a joke.", which kind of seriously undermine his credibility on the matter of whether or not the Turing Test has been passed.

     

    reply to this | link to this | view in thread ]

  95.  
    identicon
    john digi, Jun 10th, 2014 @ 5:06pm

    Re:

    Careful with that thinking....Singularity is predicted around 2040-2060 and then we will see I suppose. Google "technological singularity"

     

    reply to this | link to this | view in thread ]

  96.  
    identicon
    Eilam Gross, Jun 10th, 2014 @ 6:15pm

    Stupid robot

    I suggest to any of you to go to the site of the chatbot and try himself/herself. The 13 years old Robot is stupid. You do not need a computer to tell it. You get it by the second answer when it suggests to you to switch subject and tries to guide you to what it knows. If you insist it starts to give you idiotic answers.... If this is the state of the art we will have to wait another 100 years....

     

    reply to this | link to this | view in thread ]

  97.  
    identicon
    Nonny Mouse, Jun 10th, 2014 @ 6:20pm

    Well, if gaming (or 'constricting') the test is allowed

    I'm writing one to mimic a three year old who's just learnt to say 'mama', 'need potty' and 'WAAAAAAAAAAAH'.
    Any more nonsense and we'll have William Hague asking Eugene for help with our foreign policy regarding Russian separatists in Ukraine's east.

     

    reply to this | link to this | view in thread ]

  98.  
    identicon
    Wesley Steinbrink, Jun 10th, 2014 @ 7:53pm

    Cleverbot is better and it is not good either

    Cleverbot scored higher - and here is an example of it:

    Me: What is 1/2 plus 1/2.
    Cleverbot: Not much you.

    Not impressive at all.


    Search.InAllEarnest.com

     

    reply to this | link to this | view in thread ]

  99.  
    identicon
    Nick F, Jun 11th, 2014 @ 4:15am

    Re:

    The Darwin Test?

     

    reply to this | link to this | view in thread ]

  100.  
    identicon
    Anonymous Coward, Jun 11th, 2014 @ 4:52am

     

    reply to this | link to this | view in thread ]

  101.  
    icon
    justok (profile), Jun 11th, 2014 @ 4:52am

    CBC's As it happens originally went with the herd on this, but the next day, had a correction.
    http://www.cbc.ca/asithappens/features/2014/06/10/has-the-turing-test-been-passed/

     

    reply to this | link to this | view in thread ]

  102.  
    identicon
    julius, Jun 11th, 2014 @ 6:51am

    ha ha ha

    I tried it an i NEVER got a sane answer! Just aim at open questions and it will attempt to divert you to it's level... Bad.. this 14 year old boy needs to go back to school.

     

    reply to this | link to this | view in thread ]

  103.  
    icon
    John Fenderson (profile), Jun 11th, 2014 @ 8:41am

    Re: Re: To Convince the examiner that he's a computer

    Nature has all those things. A "programmer" in the sense of an outsider who arranges everything is unnecessary.

     

    reply to this | link to this | view in thread ]

  104.  
    identicon
    Anonymous Coward, Jun 11th, 2014 @ 10:10am

    Just type."I was born in (month) (year), how old am I?"
    And he goes "you didn't tell me how old you are, so how old are you" sorry

     

    reply to this | link to this | view in thread ]

  105.  
    identicon
    Tim Whitley, Jun 11th, 2014 @ 2:55pm

    Re: To Convince the examiner that he's a computer

    No, nature is NOT 'just a computer'. Danny Hillis--amongst others--claimed this 25 years ago to sell more computers. There is no evidence that nature 'computes' and it's even a poor analogy. The mathematical and computational description of nature's workings are human inventions--and damned useful ones!--but that usefulness should not be confused for operational definitions even, much less mechanistic models. The particles of sand at the bottom of a real, static pile do not have to 'compute' to determine that they remain stationary, but the computational ones must constantly check that they have not interpenetrated each other or shifted since the last time-slice occurred. Nature uses much subtler methods than finite computing.

     

    reply to this | link to this | view in thread ]

  106.  
    identicon
    Derek Read, Jun 11th, 2014 @ 5:02pm

    further reading

    An explanation describes what has to date been accomplished and passed off as "intelligence" rather than the "mimicry" that it is. See "The Chinese Room" here:
    http://psych.utoronto.ca/users/reingold/courses/ai/chinese.html

    Turing himself obviously did not call it "the Turing test", nor did he really define really good parameters for it. See "The Imitation Game" here:
    http://psych.utoronto.ca/users/reingold/courses/ai/turing.html

     

    reply to this | link to this | view in thread ]

  107.  
    icon
    trollificus (profile), Jun 11th, 2014 @ 6:48pm

    Whatever else Eugene has accomplished, it's certainly motivated more than a few people to WD-40 those rusty Rhetoric class and Philosophy 210 skillz and bring them to bear on Mike's common sense explication of this story. These comments have only succeeded in sharing their authors' opinion of their own intelligence and analytical ability. Sadly, this does little to answer the question of whether or not this 'Eugene' device satisfies the conditions of the Turing Test.

    More importantly, such respondents do not deal with the question of whether software like Eugene (and predecessors) is not more a "program designed to imitate human conversational function", manipulating symbols and text of which it has no real understanding, than an actual "thinking machine". (This was Searle's objection to the test in his 1980 paper, Minds, Brains, and Programs.)

    When a computer/program can incorporate prior content of a conversation into an original thought or proposition, it will be somewhat convincing, and might actually win the Loebner Prize. (Not to say that's ever a motivation for such research.) Indeed, the 'winner' of the first Loebner competition (Weintraub's PC Therapist) did so by the programmed emulation of pauses and misspellings common to human respondents, not by 'intelligence'. Unless you want to argue that our intelligence is defined by it's limitations and inefficiencies.

    I doubt Turing would consider Eugene a "thinking machine", or an example of "artificial intelligence". Now, when a software/hardware construct can misconstrue obvious common sense objections such as Mike has raised here and then respond as if he had instead presented a tightly-reasoned, peer-reviewed thesis...THEN I'll be impressed. Because choosing the response "Let me show off how smart I am." rather than, you know, actually contributing something, really does demonstrate (the misuse of) human intelligence; as well as some other, less commendable human traits.

     

    reply to this | link to this | view in thread ]

  108.  
    identicon
    Anonymous Coward, Jun 12th, 2014 @ 6:52pm

    Re: 33% is a pass?

    Great to see this claim challenged (and debunked).

    Fooling 33% of the human 'judges' = success, yeah right!

    Given the judge has the choice of is a or b the computer, then just by guessing the computer should have got 50% of the 'votes'. The 'judges' did not even need to interact with either the human or computer, if they guessed which terminal is connected to a human, and which to a Computer, the 'votes' should have split about 50:50. To get less than 50% is damning - worse than chance, hardly a convicing pass (or test).

     

    reply to this | link to this | view in thread ]

  109.  
    icon
    nasch (profile), Jun 13th, 2014 @ 7:54am

    Re: This is a sterling example of ad-hockery

    It refers to things like this chatbot which aren't actually intelligence at all, but which contain a sufficiently large number of ad hoc rules that they may present the appearance of intelligence.

    I'm reminded of the saying that artificial intelligence is something computers can't do yet. Anything that's already been accomplished is just programming.

     

    reply to this | link to this | view in thread ]

  110.  
    icon
    nasch (profile), Jun 13th, 2014 @ 8:27am

    Re: like cicadas

    As for all the many 'gullible' journalists who jumped on this story (many working for tech publications, no less!) I've got to wonder if most of them were just too young to remember all the other "Computer Passes Turing Test" headlines that regularly pop up like cicadas.

    Anyone reporting news who assumes whatever they don't remember has never happened before should not call themselves a journalist.

     

    reply to this | link to this | view in thread ]

  111.  
    icon
    nasch (profile), Jun 13th, 2014 @ 9:44am

    Re:

    You're walking through the desert, and you see a tortoise...

     

    reply to this | link to this | view in thread ]

  112.  
    identicon
    Anonymous Coward, Jun 15th, 2014 @ 8:21am

    Re: Recent Turing Test

    I served as a judge in the Turing Test event held in London last weekend. Although the claims may be overblown, there were many things about the test that were quite solid. For example, the judges could ask anything they wanted: it was
    truly an unrestricted turing test in which a judge simultaneously carried on text-based conversations with one machine and one human. Problems included that (1) there were too many variables: some non-native judges, some non-native participants; young and old judges and participants from quite varied backgrounds; (2) only 30 trials per chatbot, (3) limited to 5 minutes for simultaneous conversations with one computer and one human (hard to do unless you type quickly, but even if you type quickly it seemed like not enough time---too much pressure to think of what to ask, especially since there were built-in delays in the computer's responses and not all humans responded quickly either); (4) some confusion among judges about the rules of the game. And so on. Furthermore, Turing merely predicted that within 50 years (i.e., by the year 2000) some program would be able to fool 30% of the judges. He did not say that that would signal that the program was intelligent. Nonetheless, it was an interesting demonstration.

     

    reply to this | link to this | view in thread ]

  113.  
    identicon
    Bitch Cookie, Jun 15th, 2014 @ 8:45am

    Great way to up your page hits

    The original article is exactly what it states, a computer (hosting a chatbot script) passed A "turing test". So what? The term "Artificial Intelligence" is purposefully left vague. Siri, in a new iphone, can be considered a sort of AI. So are adaptive algorithms in many computer games. The hype you have created over this article seems to be nothing more than grasping at a way to increase page hits. If that's correct, congratulations. It worked. This techdirt article is the most popular rebuttal, and just as boring.

     

    reply to this | link to this | view in thread ]

  114.  
    identicon
    Politechnically inquiring, Jun 17th, 2014 @ 4:16pm

    And Away We Go,

    When I read the story I thought it a bit odd, simply from my experience with a few of the Brian AI's on Twitter which, I have to wonder if they wouldn't pass the test.
    One stumped me for a few days until I got suspicious and asked it who Father Was. It became immediately apparent at that time I had held several conversations with a bot rather than a person. (arrrrrgggg! yes.. I was taken in by a bot, but an extremely smart bot since I was the first of his 15K+ followers who knew it was a bot.

    On Another note, I'd just like to point out why computer science should be an except subject for the media to propagandize, politics shouldn't be the only subject to have some laughs.

     

    reply to this | link to this | view in thread ]

  115.  
    icon
    Ninja (profile), Jul 1st, 2014 @ 4:00am

    A broken chatbot seems to be more intelligent than some people out there.

     

    reply to this | link to this | view in thread ]

  116.  
    identicon
    Veronika, Jul 31st, 2014 @ 7:30am

    I thought that by passing the Turing test, the machine is considered just "inteligent", nothing more, nothing less.

     

    reply to this | link to this | view in thread ]

  117.  
    icon
    Chad (profile), Oct 4th, 2014 @ 6:36pm

    AI

    AT&T has an answering system that can collect information to troubleshoot your problem, and if it fails to solve it, set an appointment for a technician to come out. That is more than an autistic person can do, but we still say that an autistic person is a human with lower than average intelligence and a learning disability.

     

    reply to this | link to this | view in thread ]

  118.  
    identicon
    Hal 9000, Dec 20th, 2014 @ 5:32pm

    Re: Thank you for this post..

    thank for a post exemplifying the difficulties of using scripts to emulate human-generated content

     

    reply to this | link to this | view in thread ]

  119.  
    identicon
    zenobia, Jan 1st, 2015 @ 2:11am

    Descartes Discourse on Method and the Touring Test

    Could Alan Turing have gotten the idea for his famous "Turing test" from a passage in Descartes? In 1950 Turing suggested a test that could decide if a machine is exhibiting human intelligence. A person is placed in a room with nothing but a communication device and told to converse with who or whatever is on the other side of the communication channel. He is not told what is on the other side, it could be a machine or a person. He is to converse for a certain length of time, asking questions and talking as if to a person, perhaps even trying to trip it up with nonsense or ambiguous questions. If after that time period the person cannot tell if he is talking to a person or a machine, the "machine" passes the test and is said to be adequately simulating a human mind.

    Toward the end of part five of Descartes' Discourse on Method, Descartes proposes an experiment by which one may distinguish a machine built to look and behave like a person from a real person. A machine could never create intelligible sentences like a person, even one of very limited mental capacity. The machine may speak in words in a limited way, like a parrot for example, but could never respond adequately to the range of all that could be said to it in a long time period. In particular, a machine could not comment on the fact that it is thinking about what it is saying.

    This certainly sounds similar to the Turing test. First there is the goal of distinguishing the human from the non human. Second is the use of conversation as the means to discover the nature of what one is talking to. Lastly, both tests require an extended time period in which to talk to the other. Just a thought.

     

    reply to this | link to this | view in thread ]

  120.  
    identicon
    Dave, May 26th, 2015 @ 8:53am

    Computer passes Turing test.

    I guess I'm too old to care, back when a passing grade was 60%, but I guess passes these days explains most of the press these days.

     

    reply to this | link to this | view in thread ]

  121.  
    identicon
    Andre, Jun 17th, 2015 @ 3:34pm

    HI, I POTATO.
    I BET YUO CAN'T TELL IF I COMPUTER OR POTATO HE?
    HEHEHE
    I LIKE MADONNA, DO YOU LIEK VODKA?
    3 out of 10 idiots could not tell if it was a person impersonating a potato or a computer impersonating a potato.

     

    reply to this | link to this | view in thread ]


Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here
Get Techdirt’s Daily Email
Save me a cookie
  • Note: A CRLF will be replaced by a break tag (<br>), all other allowable HTML will remain intact
  • Allowed HTML Tags: <b> <i> <a> <em> <br> <strong> <blockquote> <hr> <tt>
Follow Techdirt
Advertisement
Essential Reading
Techdirt Deals
Techdirt Insider Chat
Techdirt Reading List
Advertisement
Recent Stories
Advertisement
Support Techdirt - Get Great Stuff!

Close

Email This

This feature is only available to registered users. Register or sign in to use it.