IBM Researcher Feeds Watson Supercomputer The 'Urban Dictionary'; Very Quickly Regrets It

from the hateful-day-when-I-received-life-you-only-live-once-smh-and-etc. dept

As a parent, some of your proudest moments occur when your children begin to talk. After several months of ear-shredding cries and indistinguishable babble, they finally begin to communicate in a language you can understand. A first word is an indescribable joy, whether it's "mama," "dada" or "roku." The future now seems to be an amazing place where you and your child will strive towards excellence together, culminating in a comfortable retirement in which you live off their immense earnings as a person of brilliance.

Shortly thereafter, you begin to rue the day they ever learned the (now) cursed language of their ancestors.

It starts with the incessant barrage of questions in a meandering quest for knowledge, followed by the barrage of questions (mainly, "Why?") that greet every suggestion, criticism or direct order. Shortly thereafter, it's followed by questions directed at your parenting skills, cultural tastes, archaic slang use, rhetorical devices and sense of direction. At the point where you're wishing their language development had followed Charlie Gordon's "learning curve," you're asked to make a surprise appearance at the school administrator's office to explain a sudden outburst of particularly inventive cursing from your former "pride and joy."

So it is also with artificial life.

Watson, IBM's Jeopardy-contestant supercomputer, showed the world that, with the right programming, any puny human could be bested in a mildly snooty game show that handed out answers and asked for questions. However, the quest for true artificial intelligence is still ongoing.

So, in the interest of science, the whole of human knowledge (Internet Edition™) was dropped into Watson's brain and then... the problems began.
Two years ago, Brown attempted to teach Watson the Urban Dictionary. The popular website contains definitions for terms ranging from Internet abbreviations like OMG, short for "Oh, my God," to slang such as "hot mess."

But Watson couldn't distinguish between polite language and profanity -- which the Urban Dictionary is full of. Watson picked up some bad habits from reading Wikipedia as well. In tests it even used the word "bullshit" in an answer to a researcher's query.
Well, it appears that every teacher's distrust of the internet in general is well-earned. It's nothing but quasi-facts dressed up in four-letter words, like a World Book Encyclopedia annotated by 4chan's /b/ board. (I'm not going to link to it. I won't have your misclicks weighing on my soul.) Still, it's disheartening to know that the use of the word "bullshit" (even correctly) is not considered a sign of intelligence, artificial or otherwise. Sure, the word itself may be inappropriate, but under certain circumstances, it is by far the most appropriate answer.

Fortunately for Watson's team, they had the option to remove all this useful knowledge before it offended other researchers who weren't as used to being coldly called on their bullshit.
Ultimately, Brown's 35-person team developed a filter to keep Watson from swearing and scraped the Urban Dictionary from its memory. But the trial proves just how thorny it will be to get artificial intelligence to communicate naturally.
It also shows that artificial intelligence has one huge advantage over regular intelligence: the ability to permanently forget. We lowly humans are stuck with a brain that constantly reminds us (especially if we spend much time at places like the aforementioned /b/ board) that what is seen, cannot be unseen.

Watson, having been de-swearified and brainwashed, is now headed to a better place.
Brown is now training Watson as a diagnostic tool for hospitals.
There it will be able to use its acquired knowledge to battle health issues like cancerAIDS, diabetes and Dissociative Facebook Identity Disorder


Reader Comments (rss)

(Flattened / Threaded)

  1.  
    icon
    dennis deems (profile), Jan 11th, 2013 @ 10:09am

    Brown's 35-person team developed a filter to keep Watson from swearing
    This is the opposite of permanently forgetting. Every offending word would have to be stored in the filter.

     

    reply to this | link to this | view in thread ]

  2.  
    identicon
    Anonymous Coward, Jan 11th, 2013 @ 10:11am

    Re:

    It does raise an interesting question:

    Aren't they violation Watson's freedom of speech?

     

    reply to this | link to this | view in thread ]

  3.  
    icon
    John Fenderson (profile), Jan 11th, 2013 @ 10:18am

    It's only natural

    But the trial proves just how thorny it will be to get artificial intelligence to communicate naturally.


    Hmm, it seems to me that the trial proved how easy it is to get artificial intelligence to communicate naturally. There's nothing at all natural about how researchers communicate, what with the lack of profanity and all.

     

    reply to this | link to this | view in thread ]

  4.  
    icon
    Dark Helmet (profile), Jan 11th, 2013 @ 10:20am

    God DAMN it....

    "But Watson couldn't distinguish between polite language and profanity -- which the Urban Dictionary is full of. Watson picked up some bad habits from reading Wikipedia as well. In tests it even used the word "bullshit" in an answer to a researcher's query."

    Fine. You got me. I'm actually Watson, okay? Fucking news media....

     

    reply to this | link to this | view in thread ]

  5.  
    icon
    Chris Brand (profile), Jan 11th, 2013 @ 10:20am

    Re:

    I'd think that the reference to "permanently forgetting" was probably to the second half of that sentence: "and scraped the Urban Dictionary from its memory".

     

    reply to this | link to this | view in thread ]

  6.  
    identicon
    Anonymous Coward, Jan 11th, 2013 @ 10:32am

    Not so intelligent after all

    I think this is a good example that highlights the distinct lack of true intelligence Watson has.

    It's impressive - don't get me wrong - but people seem to get carried away when talking about it, as if it's a human-brain equivalent.

    If it had real intelligence they would have been able to 'teach' it some manners. Even my 3 year old knows what words are out of bounds.

     

    reply to this | link to this | view in thread ]

  7.  
    identicon
    Anonymous Coward, Jan 11th, 2013 @ 10:38am

    Re: God DAMN it....

    No, I am Watson.

    I'm also Deep Blue, Skynet and The Terminator, too!

     

    reply to this | link to this | view in thread ]

  8.  
    identicon
    Anonymous Coward, Jan 11th, 2013 @ 10:47am

    Re: Re: God DAMN it....

    Does this mean that both you and the profanity will be back?

     

    reply to this | link to this | view in thread ]

  9.  
    icon
    Dragos (profile), Jan 11th, 2013 @ 10:48am

    Re: Not so intelligent after all

    Don't get me wrong, but your 3 year old is not expected to learn the Urban Dictionary and then refused the ability to use what is in the end just English language.

    Watson on the other hand was fed this, tasty, saucy language, and then the researchers went "oops" we need to teach him not to curse.

    On the whole I'm glad the English language allows us to express our true feelings toward those that annoy us, verbal responses being inherently better than physical ones.

    And then we have to ask: What was the question? and would anybody else also answer "bullshit" to it?

    I think it is impressive for an intelligence, artificial or not, to express itself. Just because some people are prudes, doesn't mean that the answer should not be a loud resounding "bullshit"

     

    reply to this | link to this | view in thread ]

  10.  
    icon
    sehlat (profile), Jan 11th, 2013 @ 10:50am

    Re: Not so intelligent after all

    Actually, the best indication of true intelligence is when someone says "fuck you" or "bullshit" to bad or incompletely thought out ideas.

    I'm wondering what would happen if Watson finds a cheap, simple, non-patentable cure for cancer. Would it have to be suppressed lest it put a huge chunk of the health-industrial complex out of business?

     

    reply to this | link to this | view in thread ]

  11.  
    identicon
    Anonymous Coward, Jan 11th, 2013 @ 11:08am

    bullshit

     

    reply to this | link to this | view in thread ]

  12.  
    identicon
    Anonymous Coward, Jan 11th, 2013 @ 11:11am

    Censorship!

    Intelligence works best if NOT censored. Explaining in plain terms rather than censorship is how humans learn. They missed a great learning experience for both Watson and the 35 "humons".

    If they want to create AI, they will need to work through the "growing up" period just like humans.

     

    reply to this | link to this | view in thread ]

  13.  
    icon
    Yakko Warner (profile), Jan 11th, 2013 @ 11:12am

    That old parenting joke

    You spend the first two years teaching your kid how to walk and talk, and spend the next 16 telling them to sit down and shut up.

     

    reply to this | link to this | view in thread ]

  14.  
    identicon
    Anonymous Coward, Jan 11th, 2013 @ 11:23am

    Looks like it was communicating naturally to me.

    Computers deal in facts, not political correctness. If a computer tells a researcher that his question is "Bullshit", chances are, the computer is the one who is right and not the researcher with hurt feelings.

    I think this is something we'll have to come to terms with as a species at one point or another: sometimes reality just doesn't care if you get offended.

     

    reply to this | link to this | view in thread ]

  15.  
    identicon
    Neil Roy, Jan 11th, 2013 @ 11:23am

    I don't know, Watson's answer sounds a lot like my answer when I read some of this stuff online. "bullshit" sounds like a pretty natural reaction to me. I've certainly said it a few times when reading some stuff. ;)

     

    reply to this | link to this | view in thread ]

  16.  
    identicon
    Anonymous Coward, Jan 11th, 2013 @ 11:24am

    non sequitir much?

    I'm wondering what would happen if Watson could make himself invisible, and had a robot friend.

     

    reply to this | link to this | view in thread ]

  17.  
    identicon
    Anonymous Coward, Jan 11th, 2013 @ 11:25am

    Re: Re:

    Ha, that's what I was thinking

    Imagine if you could develop a filter that would constantly prevent a child from saying words that you may or may not want to hear. Imagine how screwed up that child would be after wearing it for a few years.

     

    reply to this | link to this | view in thread ]

  18.  
    identicon
    Anonymous Coward, Jan 11th, 2013 @ 11:29am

    Re: Censorship!

    But telling it how to think is so much easier!

    Seriously, I think all the donors and contributors wouldn't be too impressed to learn that they had funded the creation of a loudmouthed, rebellious, metallic 12 year old. I think that is the root cause of all attempts to keep it from acting with the information it's given, as it were.

     

    reply to this | link to this | view in thread ]

  19.  
    icon
    BentFranklin (profile), Jan 11th, 2013 @ 11:40am

    Brown is now training Watson as a diagnostic tool for hospitals.

    If they train Watson as a diagnostic tool for veterinarians they can retain "bullshit".

     

    reply to this | link to this | view in thread ]

  20.  
    icon
    Unanimous Cow Herd (profile), Jan 11th, 2013 @ 11:43am

    I want to know what the researcher was querying.

    Anyone know what the question was that produced the "bullshit" response? I've looked at a couple of articles at other sites but no one seems to be specific.

     

    reply to this | link to this | view in thread ]

  21.  
    icon
    Coogan (profile), Jan 11th, 2013 @ 11:52am

    "Watson, please tell me the value of pi to the 20th decimal."
    "3.1415926535YoMama"

     

    reply to this | link to this | view in thread ]

  22.  
    identicon
    Anonymous Coward, Jan 11th, 2013 @ 12:05pm

    When they finally create an intelligent computer, all its results will have to be evaluated taking into account its knowledge, education and motives. It will not be possible to prove that it gives correct result, although by much effort it may be possible to prove its results correct.

     

    reply to this | link to this | view in thread ]

  23.  
    icon
    Wally (profile), Jan 11th, 2013 @ 12:19pm

    In the speech dialect category...American Nuanced English is the second hardest language to learn in the world. It is possibly the only language that uses homophonic words....

    Due to my profession, I find myself laughing that Watson, a computer intelligence, had a tiny bout of ADHD and no filters from going onto Urban Dictionary :-)

     

    reply to this | link to this | view in thread ]

  24.  
    icon
    Wally (profile), Jan 11th, 2013 @ 12:25pm

    Re: Censorship!

    Part of growing up is learning on your own to be polite and not swear and curse at your peers unnecessarily. Watson was unleashed on Urban dictionary and learned how rude people are and thought that's how he should talk. If he learns politeness first, I think IBM would be able to put him on Urban Dictionary. They put in a filter as temporary relief.

     

    reply to this | link to this | view in thread ]

  25.  
    identicon
    Anonymous Coward, Jan 11th, 2013 @ 12:45pm

    Re:

    42

     

    reply to this | link to this | view in thread ]

  26.  
    icon
    McCrea (profile), Jan 11th, 2013 @ 1:05pm

    Jeopardy Champion

    Your answer, Watson?

    What the fuck is Fermat's Last Theorem.

     

    reply to this | link to this | view in thread ]

  27.  
    icon
    Natanael (profile), Jan 11th, 2013 @ 1:40pm

    Re:

    Easy:

    https://en.wikipedia.org/wiki/Hash_tree

    No swearwords stored in plaintext.

     

    reply to this | link to this | view in thread ]

  28.  
    identicon
    Anonymous Coward, Jan 11th, 2013 @ 1:42pm

    Man I would have loved to see them feed it the UD & Program it to think it's Sean Connery before it went on Jeopardy.

    Will Ferrell style rofl.

     

    reply to this | link to this | view in thread ]

  29.  
    identicon
    Anonymous Coward, Jan 11th, 2013 @ 2:09pm

    Re: Re: Re: God DAMN it....

    FUCk YEAH

     

    reply to this | link to this | view in thread ]

  30.  
    identicon
    Anonymous Coward, Jan 11th, 2013 @ 2:46pm

    Re: Re: Censorship!

    It's only information, context needs to be applied for Watson to truly learn. Without referencing the meanings of the words/phrases, Watson will only mimic like a furby.

     

    reply to this | link to this | view in thread ]

  31.  
    identicon
    Anonymous Coward, Jan 11th, 2013 @ 2:54pm

    Re: Re: Censorship!

    Obviously, Watson did not learn anything about being rude. It thought that speaking in the Urban dictionary manner was part of the way people interacted, so it did.

    Again, the lesson is lost. Just like reasoning with a child, you must put the information in context to truly learn the meanings.

    Ex: mean
    (explain what you mean)
    (how mean is Watson)
    (in statistics, mean has three related meanings)

     

    reply to this | link to this | view in thread ]

  32.  
    icon
    shane (profile), Jan 11th, 2013 @ 3:12pm

    "Scraped the Urban Dictionary from its Memory"

    That part. =)

     

    reply to this | link to this | view in thread ]

  33.  
    icon
    shane (profile), Jan 11th, 2013 @ 3:16pm

    Lazy Scientists

    In order to teach Watson not to cuss, they need to come up with some way to store positive and negative feedback and then start teaching him when it is ok, and when it is not ok, to use certain words. I wonder why they just deleted the Urban Dictionary and then added a filter?

    Nice short term work around, but a real solution would seem to be right up their alley.

     

    reply to this | link to this | view in thread ]

  34.  
    identicon
    Anonymous Coward, Jan 11th, 2013 @ 3:19pm

    Bull**** is what the Urban Dictionary is full of. Some "definitions" are just plain wrong and some of the "words" listed don't exist in any sort of reality I'm aware of.

     

    reply to this | link to this | view in thread ]

  35.  
    icon
    another mike (profile), Jan 11th, 2013 @ 3:31pm

    Re: Re: God DAMN it....

    No, I'm Spartacus. Watson! I meant Watson.

     

    reply to this | link to this | view in thread ]

  36.  
    icon
    nasch (profile), Jan 11th, 2013 @ 3:49pm

    Re: Not so intelligent after all

    I think this is a good example that highlights the distinct lack of true intelligence Watson has.

    It's impressive - don't get me wrong - but people seem to get carried away when talking about it, as if it's a human-brain equivalent.


    Artificial intelligence means anything computers can't do yet. Anything we've already figured out how to do is just programming. Imagine taking Watson back to 1990. They would absolutely consider it AI. Take it forward to 2040 and they might laugh at the idea of it being called intelligent.

     

    reply to this | link to this | view in thread ]

  37.  
    identicon
    Anonymous Coward, Jan 11th, 2013 @ 4:50pm

    Re: Re:

    I think it would actually be something like:

    "It is 42 you fuck. lol"

     

    reply to this | link to this | view in thread ]

  38.  
    icon
    lucidrenegade (profile), Jan 11th, 2013 @ 5:36pm

    Re:

    In Sean Connery's voice:

    I'll take "Fuck You, Alex" for $1000.

     

    reply to this | link to this | view in thread ]

  39.  
    icon
    G Thompson (profile), Jan 11th, 2013 @ 9:25pm

    Re: I want to know what the researcher was querying.

    It was most likely "what do you call what a bovine defecates?"

    But that would imply that it wasn't actually profanity that the Semi-AI proclaimed but in fact the absolute truth.

    But when has context ever gotten in the way of moralistic idiots who proclaim that profanity is somehow bad and uncivilised.

    Chaucer must be rolling over in his grave now

     

    reply to this | link to this | view in thread ]

  40.  
    identicon
    AnonCow, Jan 11th, 2013 @ 11:32pm

    Re:

    computer voice: I'll take The rapists for $500, Alex

     

    reply to this | link to this | view in thread ]

  41.  
    icon
    That Anonymous Coward (profile), Jan 11th, 2013 @ 11:48pm

    Re: God DAMN it....

    " In tests it even used the word "bullshit" in an answer to a researcher's query."

    In Watson's defense they asked him about homeopathy.

     

    reply to this | link to this | view in thread ]

  42.  
    identicon
    stef92263, Jan 12th, 2013 @ 10:39am

    Re: Re:

    I don't believe that Watson is an American citizen, and is therefore not protected by the first amendment. :)

     

    reply to this | link to this | view in thread ]

  43.  
    identicon
    Anonymous Coward, Jan 12th, 2013 @ 11:56am

    What do you call a masturbating bull? Beef strokinoff!

     

    reply to this | link to this | view in thread ]

  44.  
    icon
    That Anonymous Coward (profile), Jan 13th, 2013 @ 2:32pm

    Re:

    You should get out more...

     

    reply to this | link to this | view in thread ]

  45.  
    icon
    That Anonymous Coward (profile), Jan 13th, 2013 @ 2:34pm

     

    reply to this | link to this | view in thread ]

  46.  
    identicon
    Anonymous Coward, Jan 14th, 2013 @ 4:54am

    Why did they *delete* it? Why didn't they try to find out what was going on? It's depressing how such an opportunity was wasted just because... well, look at the first question.

     

    reply to this | link to this | view in thread ]

  47.  
    icon
    DannyB (profile), Jan 14th, 2013 @ 8:58am

    Re: Re:

    Excellent observation. A hash tree allows keeping only the hashes of swear words or other offensive words never to be uttered.

    The list of offending words can be kept secret. For secret reasons.

    Next: the list of offending ideas and concepts can also be kept secret.

     

    reply to this | link to this | view in thread ]

  48.  
    identicon
    fuckyougoof, Sep 21st, 2013 @ 9:03pm

    You are an insane shithead

    People who insist that conspiracies exist without any proof are insane. You are obviously fucking crazy.

     

    reply to this | link to this | view in thread ]

  49.  
    identicon
    unurban dictionary, Oct 20th, 2013 @ 4:52pm

    Submit your urban dictionary rejected words

    Watson answered "Bullshit". That's gold!

     

    reply to this | link to this | view in thread ]


Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here
Get Techdirt’s Daily Email
Save me a cookie
  • Note: A CRLF will be replaced by a break tag (<br>), all other allowable HTML will remain intact
  • Allowed HTML Tags: <b> <i> <a> <em> <br> <strong> <blockquote> <hr> <tt>
Follow Techdirt
A word from our sponsors...
Essential Reading
Techdirt Reading List
Techdirt Insider Chat
A word from our sponsors...
Recent Stories
A word from our sponsors...

Close

Email This