Report Showcases How Elon Musk Undermined His Own Engineers And Endangered Public Safety

from the first-do-no-harm dept

For a long time now, it's been fairly clear that consumer safety was an afterthought for some of the more well known companies developing self-driving technology. That was made particularly clear a few years back with Uber's fatality in Tempe, Arizona, which revealed that the company really hadn't thought much at all about public safety. The car involved in the now notorious fatality wasn't even programmed to detect jaywalkers, and there was little or no structure at Uber to meaningfully deal with public safety issues. The race to the pot of innovation gold was all consuming, and all other considerations (including human lives) were afterthoughts.

That same cavalier disregard for public safety has been repeatedly obvious over at Tesla, where the company's undercooked "autopilot" technology has increasingly resulted in a nasty series of ugly mishaps, and, despite years of empty promises, still doesn't work as marketed or promised. That's, of course, not great for the public, who didn't opt in to having their lives put at risk by 2,500 pound death machines for innovation's sake. Every week there's new evidence and lawsuits showing this technology is undercooked and dangerous, and every week we seemingly find new ways to downplay it.

This week the scope of Elon Musk's failures on this front became more clear thanks to a New York Times piece, which profiles how corner cutting on the autopilot project was an active choice by Musk at several points in the development cycle. The piece repeatedly and clearly shows that Musk overstated what the technology was capable of for the better part of the last decade:

"As the guiding force behind Autopilot, Mr. Musk pushed it in directions other automakers were unwilling to take this kind of technology, interviews with 19 people who worked on the project over the last decade show. Mr. Musk repeatedly misled buyers about the services’ abilities, many of those people say. All spoke on the condition of anonymity, fearing retaliation from Mr. Musk and Tesla."

Musk's bravado, and the exaggeration of the sophistication of Autopilot, helped encourage some customers to have too much trust in the product or actively misuse it. Constantly pushing undercooked software and firmware updates without proper review also created safety challenges. But the article tends to focus heavily on how Musk repeatedly undermined his own engineers through stubborn decisions that undermined both overall safety and engineer expertise, like Musk's unyielding belief that full automated driving could be accomplished with just cameras, and not cameras and radar (or other detection tech):

"Within Tesla, some argued for pairing cameras with radar and other sensors that worked better in heavy rain and snow, bright sunshine and other difficult conditions. For several years, Autopilot incorporated radar, and for a time Tesla worked on developing its own radar technology. But three people who worked on the project said Mr. Musk had repeatedly told members of the Autopilot team that humans could drive with only two eyes and that this meant cars should be able to drive with cameras alone."

The article also makes it clear that employees that were overly happy to please Musk's whims only tended to make the overall quality and safety issues worse. And when employees did challenge Musk in a bid to improve quality and safety, things very often didn't go well:

"In mid-2015, Mr. Musk met with a group of Tesla engineering managers to discuss their plans for the second version of Autopilot. One manager, an auto industry veteran named Hal Ockerse, told Mr. Musk he wanted to include a computer chip and other hardware that could monitor the physical components of Autopilot and provide backup if parts of the system suddenly stopped working, according to two people with knowledge of the meeting.

But Mr. Musk slapped down the idea, they said, arguing it would slow the progress of the project as Tesla worked to build a system that could drive cars by themselves. Already angry after Autopilot malfunctioned on his morning drive that day, Mr. Musk berated Mr. Ockerse for even suggesting the idea. Mr. Ockerse soon left the company."

None of this is particularly surprising for folks who have objectively watched Musk, but it does catalog his erratic bravado and risk taking in a comprehensive way that makes all of it seem notably more concrete. For a man whose reputation is one of engineering savvy, the report repeatedly showcases how Musk refused to actually listen to his own engineers. There's little doubt Musk has been innovative, but the report does a fairly solid job showcasing how a not insubstantial portion of his near-deified reputation is more than a little hollow.

Hide this

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: autonomous vehicles, elon musk, safety, self-driving
Companies: tesla


Reader Comments

Subscribe: RSS

View by: Time | Thread


  • icon
    jvbattlewood (profile), 8 Dec 2021 @ 1:58pm

    I'm thinking that a 2,500 ton vehicle is rather a heavy for the typical American road.
    A 5,000,000 lb. vehicle is more likely to be immobile and quite safe actually, as long as you're not underneath it as it sinks into the road.

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 8 Dec 2021 @ 2:03pm

    Oh great you just activated the Douche Signal.

    I can't wait to hear from Elon and his fanbois how you're just like totally wrong about this man.

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 8 Dec 2021 @ 3:01pm

    The car involved in the now notorious fatality wasn't even programmed to detect jaywalkers,

    Industry standard. The auto industry literally invented jaywalkers, while busy stealing streets from everyone else but cars. Jaywalkers brought it on themselves, see?

    reply to this | link to this | view in chronology ]

  • This comment has been flagged by the community. Click here to show it
    identicon
    trontimouse, 8 Dec 2021 @ 4:39pm

    Elon is right about everything

    Hey, have you ever thought that the engineer was only giving the information that supports his own view of the situation? After all it is only human nature to want to be right all the time. So just as Elon Musk is not always right, so too is this engineer not right in the whole story.

    Elon has a vision and needs to make progress towards it now and not in 20 years time. Sure, things will need to be tweeked, but that vision needs to be implemented.

    Elon is a software engineer which means he gets the idea, then implements the idea, not the whole of everything. After the idea seems to work it is tweeked and tweeked until it does the thing it needs to do. Only then is the whole rest of the program added, things like safety and security. That is the part where the software works as advertised but needs to be made safe.

    As you can see, Elons software as is everyone else's , is not finished and making it into a finished product is not yet being worked on. Elon is still iterating and so are the engineers. So eventually when the auto pilot is doing its job properly , then the software will be added to make it safe and secure. That day is not yet here, so caution when using auto pilot or any other self driving software.

    reply to this | link to this | view in chronology ]

    • icon
      danderbandit (profile), 8 Dec 2021 @ 5:09pm

      Re: Elon is right about everything

      As you can see, Elons software as is everyone else's, is not finished and making it into a finished product is not yet being worked on.

      I was looking for the /s but you appear to be serious. If the car is not a finished product what are those that I see driving around?

      What you are saying, whether you intended to or not, is the buyers are no more than beta testers. And paying handsomely for the privilege.

      reply to this | link to this | view in chronology ]

      • icon
        OldMugwump (profile), 9 Dec 2021 @ 8:11am

        Re: Re: Elon is right about everything

        Imperfect as Autopilot is (and it is imperfect; I have 2 Teslas), it appears to be nonetheless safer than the average human driver.

        The bottom line is number of accidents per mile driven - Teslas generally, and Autopilot specifically, have less accidents that average cars.

        Ref: https://www.tesla.com/VehicleSafetyReport

        So the article and Musk attacks seem pretty unfair - sure, nothing's perfect, but surely "better" is an improvement to be praised.

        reply to this | link to this | view in chronology ]

        • icon
          James Burkhardt (profile), 9 Dec 2021 @ 9:15am

          Re: Re: Re: Elon is right about everything

          Crashes are kinda fun, data-wise.

          We used to have super rigid car frames, like tesla. Eventually we realized having flex and crumple built in reduced injuries. Societally we choose to inflict more severe damage to a car over injuries to passengers. The move to a super durable frame might radically shift us back to physical injuries as the norm.

          The length of a yellow light affects the rate of crashes in city driving. Longer yellow lights reduces reds being run, and reduces sideswipes but comes with an increased risk of rear-endings. Car accidents go up...seriousness of the accidents goes down. We can trade fewer deadly accidents for more accidents with minor injuries and damages.

          This is to say focusing on the number and frequency of accidents might be a bad measure. Autopilot's tendancy to hit parked emergency vehicles obstructing the road for instance might be a much more serious error than the error the human makes merging to avoid the vehicle.

          Relying on a self reported black box accident report might not highlight the whole picture. For instance, Autopilot would I expect be most used on freeways were traffic conditions are more stable, and distance is most common. But the most time spent driving was on city streets, where all autonomous driving tech is struggling to go beyond a non-scalable system of heavily-mapped isolated regions. Most accidents happen on city streets. So the sample of all crashes from automobiles might not map well to the sample of Autopilot drivers. Indeed, given that without autopilot tesla drivers commit fewer accidents than the national average likely means a higher attention to the road by drivers in general, and a strong possibility that drivers have been over-riding Autopilot to avoid accidents. Absent data on overrides, I can take Tesla's data and build a much less glowing perspective on safety.

          Without active safety features, Tesla drivers encounter accidents a third less often than the average driver who may or may not have active safety features. This suggests that Tesla drivers are in general more attentive to the road. Given that many Autopilot errors are not ones an attentive human would have made, this suggests that the accidents we see likely have less attentive drivers. That suggests Tesla's active safety features only have the benefit they do when an attentive driver is available to over-ride Autopilot.

          Right now, most people dont have and can't afford a Tesla w/ Autopilot. Once that changes, the drivership of Tesla cars will likely become less attentive, not more, as the drivership approaches the societal mean reflected in the national average. That means as it stands right now, we might actaully be seeing a less safe technology whose safety is being artificially boosted by current drivership.

          reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 8 Dec 2021 @ 5:24pm

      Re: Elon is right about everything

      Trying to parse what you just said and I cannot tell if you are being sarcastic or not...

      Is what you are trying to say is the software is not yet and is not yet ready to be released from beta testing to production. I personally do not want to be part of this beta test where when something goes wrong lives are on the line.

      reply to this | link to this | view in chronology ]

      • identicon
        OGquaker, 8 Dec 2021 @ 8:09pm

        An A.C. is right about everything

        A dozen new high-rise buildings in Los Angeles were abandon and left vacant for 20 years after the 1994 earthquake, not because they failed, but because the City bothered to inspect the structures. See https://www.npr.org/2021/11/29/1059796996/the-story-of-the-boeing-737-max

        reply to this | link to this | view in chronology ]

        • icon
          James Burkhardt (profile), 9 Dec 2021 @ 9:27am

          Re: An A.C. is right about everything

          Could I get a time stamp on the high-rise discussion? I don't have 47 minutes to go back over the boeing 737 max debacle for a tangentally related 4 minute segment on LA high-rises.

          reply to this | link to this | view in chronology ]

      • icon
        Clandestine (profile), 9 Dec 2021 @ 7:44pm

        Re: Re: Elon is right about everything

        Well, then do not use the auto-pilot feature. Just drive it like the rest of us. However, you might feel unsafe using the auto-pilot, but I'd feel safer in my standard car if you were not driving yours manually because it is much more likely that you will make a mistake and kill me than your car killing me.

        reply to this | link to this | view in chronology ]

    • icon
      jvbattlewood (profile), 8 Dec 2021 @ 5:41pm

      Re: Elon is right about everything

      If architects designed structures the same way software engineers write code; the first woodpecker that came along would destroy civilization.

      reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 8 Dec 2021 @ 6:37pm

      Re: Elon is right about everything

      After the idea seems to work it is tweeked and tweeked until it does the thing it needs to do. Only then is the whole rest of the program added, things like safety and security. That is the part where the software works as advertised but needs to be made safe.

      Autopilot doesn't "work as advertised" until it's safe to use. One doesn't simply design an unsafe system and only later "add" security and safety. Safety isn't a substance or object that can be taped onto a product in the final stages of development. To expect otherwise would be like trying to make ice cream healthier by adding vegetables after the ice cream has already been made. Security and safety should be continuously evaluated throughout the entire design process.

      reply to this | link to this | view in chronology ]

      • identicon
        Anonymous Coward, 8 Dec 2021 @ 6:39pm

        Re: Re: Elon is right about everything

        Previous commenter here. Meant to reply to trontimouse's original comment but clicked the wrong button.

        reply to this | link to this | view in chronology ]

      • identicon
        OGquaker, 8 Dec 2021 @ 9:33pm

        Neal Boudette lives in Detroit, no suck-up there

        Tesla's auto industry veteran Right; a Buick engineer lamented to me that they were ready with airbags 10 years before Detroit relented, my Mom's 1962 Grand Prix had seat-belt anchors under the carpet (I crawled under and saw the welded nuts) but GM failed to sell belts (patented in 1885) until 1964. See 'Tucker: The Man and His Dream' (1988, produced by George Lucas) Plot Point: Apr 21, 2008 purposely did subpar work for Tesla and then stole trade secrets, Tesla sued last week in a California court.

        this beta test where when something goes wrong lives are on the line What planet are you from? Detroit, Super Markets, Pringles, Fast food, Nutrasweet, Dr. Foutche beta test on everyone, the hepatitis vaccine was "beta tested" on thousands of people (RIP) around the world from 1969 until a New version stopped killing people in 1983. The CDC sent my healthy Mother a letter, suggesting a trial, she May have been exposed to hepatitis from a bad hotel sewer the letter said ... She was dead by 1981: "Acquired Immune Deficiency"
        December 8, 2021 16:50 GMT: https://www.reuters.com/business/autos-transportation/renault-zoe-goes-hero-zero-european-safety-age ncy-rating-2021-12-08/ Biden's PAYING Detroit to make electric cars in Mexico, but nothing for Tesla: they paid back their Federal loan WITH INTEREST many years ago.

        reply to this | link to this | view in chronology ]

        • identicon
          Anonymous Coward, 8 Dec 2021 @ 10:00pm

          Re: Neal Boudette lives in Detroit, no suck-up there

          Let's hear it for incoherent screeds, and links with no relevance to the topic at hand OR the screed.

          reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 8 Dec 2021 @ 6:59pm

      Re: Elon is right about everything

      "Elon is a software engineer which means he gets the idea, then implements the idea,"

      er, since when?

      i think you mean 'elon is an immature richboi, which means he gets an idea, then people try to fulfil his crazy idea while he sits back and acts like he does all the work'.

      He 'has ideas' (or 'visions')
      The only difference between him and the person gibbering outside the local methadone dispensory is his dad was wealthy, and he leveraged that into having himself viewed as someone smart and productive, despite him never doing much either productive or smart except taking credit. The only real difference between him and Trump? 1302 weeks.

      reply to this | link to this | view in chronology ]

      • identicon
        Anonymous Coward, 9 Dec 2021 @ 4:08am

        Re: Re: Elon is right about everything

        The only real difference between him and Trump?

        The ability to hire competent people as opposed to more frauds.

        reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 8 Dec 2021 @ 8:27pm

      Re: Elon is right about everything

      Hi Elon!

      reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 8 Dec 2021 @ 11:11pm

      Re: Elon is right about everything

      Or maybe, just maybe...

      That autopiloting software in cars need to actually take into account safety and the fact that there are HUMAN LIVES in the equation?

      It's not a damn videogame, for crying out loud.

      reply to this | link to this | view in chronology ]

    • icon
      Scary Devil Monastery (profile), 9 Dec 2021 @ 12:44am

      Re: Elon is right about everything

      "Elon has a vision and needs to make progress towards it now and not in 20 years time. Sure, things will need to be tweeked, but that vision needs to be implemented."

      The issue being that Elon cutting corners on safety means that vision comes at the cost of investments - and possibly lives - which weren't volunteered for that purpose.

      "Elons software as is everyone else's"

      It really isn't. Software which may endanger lives is not to be issued in beta version.

      At the end of the day if the result of the means is casualties the end goal is ruined. This is how you turn a laudable vision into a failed pie-in-the-sky project. By using means which result in the end goal never being reached.

      reply to this | link to this | view in chronology ]

    • icon
      JoeCool (profile), 9 Dec 2021 @ 2:50am

      Re: Elon is right about everything

      If it's not safe and secure, it should be on a test track, not city streets. That's why we test drivers and their cars - to keep the public safe. It's why the cops look for unsafe driving or people breaking the rules (or at least it is SUPPOSED to be why they're out on the roads). Until the automated cars are proven to be at least as safe as a human driver, they shouldn't be on public roads.

      reply to this | link to this | view in chronology ]

      • identicon
        Anonymous Coward, 12 Dec 2021 @ 11:17am

        Re: Re: Elon is right about everything

        You realize GM, NISSAN Ford etc have 8-9 TIMES the number of fatal accidents caused by poor engineering right?

        so by your proposal, we should take all GM, Nissan, Ford etc cars off the streets post haste?

        reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 8 Dec 2021 @ 10:09pm

    humans could drive with only two eyes and that this meant cars should be able to drive with cameras alone.

    ... and a quite remarkable analytic engine driving those eyes and those cars.

    When Musk produces a car that can greet me by name, despite any changes I might make to my appearance, I might agree that his car MIGHT be able to drive with just cameras alone. I might even look to buy a kit conversion for my car.

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 9 Dec 2021 @ 6:29am

    I guess Karl Bode thinks Elon Musk needs a meat thermometer.

    reply to this | link to this | view in chronology ]

  • icon
    Quarthinos (profile), 9 Dec 2021 @ 7:04am

    Lack of research from Mr Bode showing through again.

    A quick google search reveals the weight of a Tesla is between 3500 and 4500 pounds, not 2500 pounds (or tons? LOL!)

    Is techdirt just an SEO platform for the NYT these days?

    reply to this | link to this | view in chronology ]

    • icon
      James Burkhardt (profile), 9 Dec 2021 @ 9:48am

      Re: Lack of research from Mr Bode showing through again.

      Your fallacy is....Ad-hominem

      You've dismissed the actual discussion at hand by seizing on Mr. Bode's use of an outdated industry average. I wanted to say you were implementing the fallacy fallacy, that you picked up a minor error that actually underplayed Karl's point, and made like that renders the article meaningless.

      But you didn't even try to make that silly argument, instead just making an ad-hom attack. Makes it easier to choose not to explain how both those errors came about, since you clearly don't care.

      reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 9 Dec 2021 @ 1:00pm

      Re: Thanks for creating an account though

      You know that Elons not gonna touch it no matter how much you flatter him, right?

      reply to this | link to this | view in chronology ]

    • icon
      K`Tetch (profile), 9 Dec 2021 @ 1:06pm

      Re: Lack of research from Mr Bode showing through again.

      2500lb is a well accepted 'gut value' for a car. I think the actual average weight these days of new cars is 2800lb, but when you consider many older cars, it brings the weight down (my current car is 2350lb, I've had two volvos that were 2180lbs (the glorious 300 series), and a 140mph 4-seat car (MG Metro twin-turbo) that was 1850lb (including the extra engine stabilizers, intercooler, and a Citroen active antiroll suspension)

      so yes, as a shorthand for car "2500lb death machine" is a well accepted common use reference to both average weight, and how easy it is for it to kill.

      But if that's the ONLY criticism you could manage to actually level (that you don't understand idioms), says a lot about how good the article was.

      reply to this | link to this | view in chronology ]

    • icon
      techflaws (profile), 11 Dec 2021 @ 3:06am

      Re: Lack of research from Mr Bode showing through again.

      Which makes Karl's point about death machines even more appropriate, no?

      reply to this | link to this | view in chronology ]

    • icon
      Scary Devil Monastery (profile), 13 Dec 2021 @ 3:04am

      Re: Lack of research from Mr Bode showing through again.

      "Is techdirt just an SEO platform for the NYT these days?"

      Says the shill who built an account just to post two comments in defense of Musk.

      I note you didn't dispute anything of actual relevance in the OP.

      reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 10 Dec 2021 @ 2:28am

    Take out all bias from this shit-post and you are left with jealous clickbait with stats from 2015.

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 12 Dec 2021 @ 11:16am

    I suspect Karl Bode should probably have checked into just how much GM, Ford etc put into Advertising in the NY Times, and how often they've previously been caught in smear campaigns against their the rivals of their biggest advertisers.

    Ya know, for balance in journalism instead of parroting car-company fluff.

    reply to this | link to this | view in chronology ]


Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here



Subscribe to the Techdirt Daily newsletter




Comment Options:

  • Use markdown. Use plain text.
  • Make this the First Word or Last Word. No thanks. (get credits or sign in to see balance)    
  • Remember name/email/url (set a cookie)

Close

Add A Reply

Have a Techdirt Account? Sign in now. Want one? Register here



Subscribe to the Techdirt Daily newsletter




Comment Options:

  • Use markdown. Use plain text.
  • Make this the First Word or Last Word. No thanks. (get credits or sign in to see balance)    
  • Remember name/email/url (set a cookie)

Follow Techdirt
Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Discord

The latest chatter on the Techdirt Insider Discord channel...

Loading...
Recent Stories

This site, like most other sites on the web, uses cookies. For more information, see our privacy policy. Got it
Close

Email This

This feature is only available to registered users. Register or sign in to use it.