Online Search For Fossett A Waste Of Time?

from the needle-in-a-haystack dept

Back in September, after millionaire Steve Fossett went missing in the Nevada backwoods, legions of web surfers enlisted to help in the search through Amazon's Mechanical Turk. The 50,000 volunteers looked through hundreds of thousands of satellite images of the area for any clues of Steve Fossett's plane. The whereabouts of Steve Fossett are still unknown, and now some are starting to question whether or not Mechanical Turk was a help or a hindrance during the search. Members of the search party, when fed search hints from the online volunteers, complained that the leads were false and ended up wasting their time. Sure, some poor leads may have been passed to search teams, but many important lessons were learned during this process that can be applied to future uses of this tool. So, hopefully search professionals will not be quick to attribute the outcome of the search to the use of Mechanical Turk -- after all, traditional methods to find Fossett were also unsuccessful. So, by understanding these tools better, a success surely lies in the future.


Reader Comments (rss)

(Flattened / Threaded)

  1.  
    identicon
    kilroy, Nov 7th, 2007 @ 10:34am

    but

    If legions of web-surfers had joined in the physical search would we have found him?

     

    reply to this | link to this | view in thread ]

  2.  
    identicon
    RoundSquare, Nov 7th, 2007 @ 10:47am

    How about using the Turk to help people find their way:

    Seeing eye people

     

    reply to this | link to this | view in thread ]

  3.  
    identicon
    Anonymous Coward, Nov 7th, 2007 @ 11:33am

    Re: but

    Maybe, but how many of those millions had time or could afford to take a trip to partake in the search?

     

    reply to this | link to this | view in thread ]

  4.  
    identicon
    Anonymous Coward, Nov 7th, 2007 @ 12:11pm

    Wrong application of a great tool?

    Crowdsourcing might work great for tasks like tagging easily identifiable images, but it seems like an impossible expectation was created by assuming that laypersons could identify a crash site from these images. This, coupled with the participants' desire to "be a hero" (which is understandable and should have been anticipated) may have squandered some valuable resources with false positives. But it's hard to complain about this as a "first effort." The crowd's hearts were in the right place.

    The article said that each flagged image was subjected to at least 10 reviews. I expect they would have gotten fewer false positives if they had a mechanism to make suspected "hits" bubble to the top of everyone else's queue--and no action should be taken on a lead without broader community concensus.

    The early success of Google as a search engine was often attributed to innovative algorithms that allowed the 'web' to collectively 'vote' for the top results (i.e. by counting links back to a page). I would think this concept should be one of the cornerstones of any crowdsourcing effort where the participants have a singular, common goal.

     

    reply to this | link to this | view in thread ]

  5.  
    identicon
    Sean, Nov 7th, 2007 @ 12:28pm

    Why

    If they had these images to look for him why did they not also get the most recent images of the area before the crash overlay them and have a program look for all and any changes between the two? Then have humans look over the flaged changes for actuall evidence that there was a crash. That would be more precise and take less time.

     

    reply to this | link to this | view in thread ]

  6.  
    identicon
    Anonymous Coward, Nov 7th, 2007 @ 12:53pm

    Re: Why

    I suspect that technology is a bit too cutting edge. Its probably only used by military right now. Its harder than you think.. filtering out clouds and unexpected shadows and things is not easy. But your idea is sound, and it will eventually find its way into the public's toolchest

     

    reply to this | link to this | view in thread ]

  7.  
    identicon
    Geoff, Nov 7th, 2007 @ 1:20pm

    question

    Did anyone check for Fossett under the goal posts at Giant's Stadium?

     

    reply to this | link to this | view in thread ]

  8.  
    identicon
    Joe Smith, Nov 7th, 2007 @ 1:34pm

    Let's not forget

    that apparently the crowd was able to correctly identify multiple previously unknown crash sites all of which had presumably been searched for by traditional methods.

     

    reply to this | link to this | view in thread ]

  9.  
    identicon
    CharlieHorse, Nov 7th, 2007 @ 1:57pm

    Let's Also Not Forget

    that the images that were tagged as possibles by volunteers online were not just sent straightaway to the search teams. the images that were tagged were then looked at by professional analysts and THEN they were passed on to the search teams. so, I'd actually argue that the onus is not on the "online volunteers" - but on the pros who must have agreed to a large degree with the "amateur" online volunteers ...

    all told, especially considering comment #8 - and Joe is 100% spot on in bringing up this point - and I think it's the real issue here - and that is: while the online volunteers did not (or have not as yet) found the Fosset wreckage - they sure did find a lot of other wrecks- so I think that alone validates using the crowd-sourced approach to problems of this nature - and this problem in particular.

     

    reply to this | link to this | view in thread ]

  10.  
    identicon
    Adrian Blakey, Nov 7th, 2007 @ 2:17pm

    When he's found

    Hopefully someone will go back and look at the images and see if anything could have been found, and whether or not the specific images had been viewed by a "turk."

     

    reply to this | link to this | view in thread ]

  11.  
    identicon
    Anonymous Coward, Nov 7th, 2007 @ 4:35pm

    Re: Let's Also Not Forget

    the images that were tagged were then looked at by professional analysts and THEN they were passed on to the search teams
    Not always. Apparently some of the volunteers were so hungry for attention and fame that they took it upon themselves to bypass the system and started e-mailing and phoning Civil Air Patrol searchers directly with a flood of bogus leads.

    while the online volunteers did not (or have not as yet) found the Fosset wreckage - they sure did find a lot of other wrecks
    Really? How many? What's your source? The article says otherwise in that the volunteers turned up nothing productive.

    so I think that alone validates using the crowd-sourced approach to problems of this nature - and this problem in particular.
    Just the opposite. How is wasting a lot of the searchers time with a lot of bogus leads a "validation"? (unless that's your objective)

     

    reply to this | link to this | view in thread ]

  12.  

    50,000 volunteers. Is this a record?

    The volunteers have to be commended. A think a lot will be learned by this effort and this technology can only get better with time.

    I still can't believe this happened to Fossett. Does anybody think he could still be alive?

     

    reply to this | link to this | view in thread ]

  13.  
    identicon
    Anonymous Coward, Nov 7th, 2007 @ 5:03pm

    Re: Wrong application of a great tool?

    This, coupled with the participants' desire to "be a hero" (which is understandable and should have been anticipated) may have squandered some valuable resources with false positives.
    I think they really under estimated the desire that some people have for fame and attention. This desire will lead some people to do almost anything to get it (e.g. shoot up school). What's to keep an attention starved individual from just flooding the system with bogus reports in the hope that one of them might accidentally hit? This is the way some "psychics" operate. They make tons of predictions, most of which are wrong, in the hope that they'll accidentally get one right. They quickly forget all their failures and publicly grandstand on the very few they guess right.

    The crowd's hearts were in the right place.
    You're making the mistake of assuming that the crowd was of one heart. Some may have been doing it for noble reasons. Others may have just been doing it for a chance at fame (which is not noble).

    I would think this concept should be one of the cornerstones of any crowdsourcing effort where the participants have a singular, common goal.
    How do you ensure that the participants have "a singular, common goal"? The lesson here is that if you want quality results you need some kind screening system for the participants.

     

    reply to this | link to this | view in thread ]

  14.  
    identicon
    Anonymous Coward, Nov 7th, 2007 @ 5:13pm

    Re: Wrong application of a great tool?

    The article said that each flagged image was subjected to at least 10 reviews.

    No, it didn't. It said just about the opposite in that the images were subject to "up to" 10 reviews (which includes zero). "At least" includes infinity. Zero and infinity are nowhere the same. Neither are "up to" and "at least".

    You don't by chance work as a salesperson for a large ISP or telco, do you?

     

    reply to this | link to this | view in thread ]

  15.  
    identicon
    Anonymous Coward, Nov 7th, 2007 @ 5:18pm

    Re: Let's not forget

    that apparently the crowd was able to correctly identify multiple previously unknown crash sites all of which had presumably been searched for by traditional methods.
    I didn't see that in the article. It seemed to say the opposite when I read it.

     

    reply to this | link to this | view in thread ]

  16.  
    identicon
    Anonymous Coward, Nov 7th, 2007 @ 5:29pm

    Re: Why

    why did they not also get the most recent images of the area before the crash overlay them and have a program look for all and any changes between the two?
    Because there will always be differences between them due to time of day, weather, position of observer, natural forces on the ground, etc. Thus all the images would be flagged. That's not really very helpful, now is it?

     

    reply to this | link to this | view in thread ]

  17.  
    identicon
    CharlieHorse, Nov 7th, 2007 @ 8:55pm

    RE: Let's also not forget

    "Not always. Apparently some of the volunteers were so hungry for attention and fame that they took it upon themselves to bypass the system and started e-mailing and phoning Civil Air Patrol searchers directly with a flood of bogus leads."

    Fine, that may be the case - but it just further proves my argument. The onus is then REALLY on the CAP to ignore or process those results via their "professional" methodology.

    Really? How many? What's your source? The article says otherwise in that the volunteers turned up nothing productive.

    30 seconds found these two articles. I had remembered reading similar during the search effort.

    http://www.nytimes.com/2007/10/04/us/04fossett.html?_r=1&oref=slogin
    http://news.bbc .co.uk/2/hi/americas/6986453.stm

    thus, I would still argue that the results clearly validated, or demonstrated if you like, that, in this type of problem, a crowd-sourced approach to problem solving is quite effective.

     

    reply to this | link to this | view in thread ]

  18.  
    identicon
    Anonymous Coward, Nov 7th, 2007 @ 9:41pm

    Re: RE: Let's also not forget

    The onus is then REALLY on the CAP to ignore or process those results via their "professional" methodology.
    I expect that ignoring online volunteers is exactly what the CAP intends to do in the future. And for good reason.

    30 seconds found these two articles. I had remembered reading similar during the search effort.
    http://www.nytimes.com/2007/10/04/us/04fossett.html?_r=1&oref=slogin
    http://news.bbc .co.uk/2/hi/americas/6986453.stm
    Neither of those stories credit the finds to online searchers. What, you thought no one would check your sources? Think again.

    thus, I would still argue that the results clearly validated
    And I would now label you as intellectually dishonest due to your little stunt above.

    Now everyone imagine what a lot of similarly dishonest online "volunteers" could do to such an effort an I think my point is made.

     

    reply to this | link to this | view in thread ]

  19.  
    identicon
    CharlieHorse, Nov 8th, 2007 @ 3:52pm

    okay, for the last time -

    You have conceded that I am correct in my first point. The onus is on the CAP. Therefore, we agree on this point.

    as for my second argument, I would certainly hope that others will check/do the research themselves. if you/they don't, then THAT's intellectually dishonest.

    as for your personal feelings towards me - they are irrelevant to the discussion at hand. you may label me anything you like - it sill does not change the facts:

    as I stated, I had recalled reading that during the s&r op - numerous other wrecks had been found. you asked for sources. I provided them. numerous other wrecks had, in fact, been found. even if not a single wreck was found by the clues fed to the search teams - there is more:

    this link provides an interesting bit of info on the impact the volunteers had on the search (yes, found in about another 30 second search):
    http://mashable.com/2007/09/14/steve-fossett

    note that the volunteers were able to narrow the search area significantly. (you will recall the area being searched was on the order of size of a small US state). this, in itself, again validates using a crowd-sourced approach. more eyes on the images is A Good Thing - especially when time is a critical factor. the simple fact that the search area was narrowed leads to the reasonable assumption that the volunteers must have been providing at least some good data.

    so, once again, in response to the question posited by the Dennis Yang - whether or not crowd-sourced solutions were a "waste of time", i.e. a useful endeavor - it is my continued assertion that crowd-sourcing problem solving approaches to problems of this nature are quite effective and useful.

    Are there important lessons to learn from the fossett search effort? - of course there are. But let's not throw out the proverbial baby with the bathwater here. the problems faced in this search can, and should, be worked out.

    best wishes,
    Charlie.

     

    reply to this | link to this | view in thread ]

  20.  
    identicon
    Anonymous Coward, Nov 8th, 2007 @ 4:09pm

    Re: okay, for the last time -

    You have conceded that I am correct in my first point. The onus is on the CAP. Therefore, we agree on this point.
    Yep. To keep your butt out of things.

    as I stated, I had recalled reading that during the s&r op - numerous other wrecks had been found. you asked for sources. I provided them.
    You stated that the "online volunteers" found "a lot of other wrecks" and then provided bogus references for that claim. Anyone can look above to verify this for themselves so the only person you may be fooling is yourself. You have new crossed from the realm of intellectually dishonest to that of plain old liar. A bunch of "volunteers" like you could sabotage any search project.

     

    reply to this | link to this | view in thread ]

  21.  
    identicon
    Something is not right, Nov 11th, 2007 @ 10:11pm

    How do we know that he didn't plan to disappear? No one knows what he was doing flying, first they say to scout a test track (which turns out was already picked out), later to sightsee. Which he was there before, it wasn't his first time in the area. Then they said he had a special watch, later they said he wasn't wearing it. He didn't even tell anyone what direction he was flying. More people know my life and if I leave my home where I am going.

     

    reply to this | link to this | view in thread ]

  22.  
    identicon
    Anonymous Coward, Oct 2nd, 2008 @ 11:24am

    Now that the plane has been found, obviously we should re-examine those photos and see whether anything could have been detected by a typical viewer or not.

     

    reply to this | link to this | view in thread ]


Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here
Get Techdirt’s Daily Email
Save me a cookie
  • Note: A CRLF will be replaced by a break tag (<br>), all other allowable HTML will remain intact
  • Allowed HTML Tags: <b> <i> <a> <em> <br> <strong> <blockquote> <hr> <tt>
Follow Techdirt
A word from our sponsors...
Essential Reading
Techdirt Reading List
Techdirt Insider Chat
A word from our sponsors...
Recent Stories
A word from our sponsors...

Close

Email This