57 votes

Tesla’s “self-driving” system never should have been allowed on the road

Topic removed by site admin

37 comments

  1. [3]
    miyu
    (edited )
    Link
    I get people don't like hearing this, but... this article is over ~17 deaths since 2021 (2 years!) when ~43000 people die annually of car crashes within the US. It's peanuts. If you believe...
    • Exemplary

    I get people don't like hearing this, but... this article is over ~17 deaths since 2021 (2 years!) when ~43000 people die annually of car crashes within the US. It's peanuts. If you believe autonomy saves lives in the long-run, then the question is how do we get to autonomy and is doing so worth it. I don't think autonomy can be reached without true miles driven at scale, so I don't see a great alternative. And I do see evidence the approach is working and AV companies are greatly improving performance over time... as the saying goes, let's not let perfect be the enemy of good.

    Anyways, this article is flawed for a few reasons:

    1. It divides the numerator "cumulative fatalities encountered by Autopilot + FSD since from sometime 2021 to mid-2023" (17 fatalities) by denominator "total miles driven by FSD since ~2021" (150m miles). The numerator doesn't match the denominator.

    To explain: Autopilot is shipped standard in all Tesla vehicles. FSD is a $15000 upgrade and has a low ~8% take-rate. That low take-rate cannot simply be multiplied against Autopilot's miles driven, because the systems have different use-cases (e.g. ability to turn on a street) and FSD has not been available for all its purchasers until recently.

    So, we don't have a great denominator. We can find a lower-bound: In 2020 AP had 5 billion km driven, up to 50 billion km by mid-2022 ((source)[https://thedriven.io/2023/01/16/the-rise-of-the-machines-tesla-drives-50km-autonomously-through-heavy-la-traffic/]). Handwaving, that's a denominator of 45 billion kilometers, or 28 billion miles.

    Note 28b is a factor of ~20 off from 150m.

    Dividing 17 fatalities by 28 billion miles gets us a fatal accident rate of autopilot of 0.0607 deaths per 100 million miles, compared to the NHTSA rate of 1.35 or the article's claimed 11.3. Now both the numerators and denominators match, relatively.

    So, doing the math 'more correctly-ish', we would get an upper bound accident rate that far less than the overall fatal accident rate as reported by the NHTSA...

    Except this is still comparing "miles within autopilot including 5 seconds after disengagement" as compared to "total miles driven by non-autopilot cars" which is definitely still apples-to-oranges, as regular drivers do not use autopilot for all driving.

    And once again, this comparison is about autopilot+human+Tesla vs regular humans in other cars. FSD expands to more use-cases and is a total rewrite of the system that is far more beta. Other cars includes older cars which might not have great active safety mechanics. Tesla also has the advantage of an instantaneous induction/PM motor. There are so many confounding factors that make it more than "is AP/FSD good or bad"...

    1. The article cites MKBHD's video where FSD supposedly struggles on the highway. On Marques's build, FSD was not running on the highway; the 5-year-old Autopilot stack was running. Older builds of FSD, switched back to legacy autopilot nearing highways.

    2. The article makes numerous other broken comparisons, e.g. making a large claim that Teslas account for 91% of self-driving crashes in NHTSA data implying that Teslas are then radically more unsafe. Teslas are also, somewhere around 91% of the fleet in terms of miles driven, give or take a few %. What's the point? Is there a real claim here?

    (Full Disclosure: I have a financial stake in TSLA)

    25 votes
    1. BitsMcBytes
      Link Parent
      This comment should be much higher. When you actually do comparative analysis, it looks much more like autopilot is reducing accidents, even though some still happen every 100 million miles. But...

      This comment should be much higher. When you actually do comparative analysis, it looks much more like autopilot is reducing accidents, even though some still happen every 100 million miles.

      But Jameson can’t continue to call Spider-Man a menace while also acknowledging a reduction in crime.

      8 votes
    2. gf0
      Link Parent
      FSD self-filters its statistics by only being applicable in good conditions, doesn’t it? So what it should be compared against is human drivers in good conditions, which is surely a much much...

      FSD self-filters its statistics by only being applicable in good conditions, doesn’t it? So what it should be compared against is human drivers in good conditions, which is surely a much much lower fatality rate.

      I believe it is also fair to assume that many (even most?) fatal accidents are caused by criminally irresponsible drivers (DUI) making impossible situations - and I am not convinced that these are handled better by AIs.

      So unless you could replace the whole of humanity with self-driving cars (which is just unrealistic), I don’t think that we would see good, if any improvement from an incremental replacement of human drivers from the car pool. We would get a much much better bang for bucks by simply making lane assist/emergency break at slower speeds/etc features standards/requirements which are happening and is actually a realistic goal - as these are things where human reaction time is indeed the limiting factor. This is my gripe with self-driving (besides many selling snakeoil)

      1 vote
  2. [3]
    unkz
    (edited )
    Link
    According to the Post the crashes were using autopilot rather than FSD. It sounds like autopilot is vastly wider deployed than FSD, so the math here doesn’t seem to quite add up. From 2020,...

    According to the Post the crashes were using autopilot rather than FSD. It sounds like autopilot is vastly wider deployed than FSD, so the math here doesn’t seem to quite add up. From 2020,

    https://lexfridman.com/tesla-autopilot-miles-and-vehicles/

    Estimated Autopilot miles to-date: 3.3 billion miles

    20 votes
    1. TAn0n
      Link Parent
      The Washington Post article notes that the NHTSA does not distinguish between Autopilot and Full Self-Driving systems in their data. SOURCE: WP article Tesla sold over 500,000 Model 3s in 2021...

      The Washington Post article notes that the NHTSA does not distinguish between Autopilot and Full Self-Driving systems in their data.

      It is unclear which of the systems was in use in the fatal crashes: Tesla has asked NHTSA not to disclose that information. In the section of the NHTSA data specifying the software version, Tesla’s incidents read — in all capital letters — “redacted, may contain confidential business information.”

      SOURCE: WP article

      Tesla sold over 500,000 Model 3s in 2021 alone. Probably over a million vehicles since 2019. All of them come with Autopilot.

      This "article" is terrible sensationalism. And this from someone that despises Musk and will never own a Tesla.

      16 votes
    2. Autoxidation
      Link Parent
      According to Tesla's Vehicle Safety Report: It's more than 9 billion miles by now, though I'm unsure of "when" is. The page was last updated in Jan 2023 at least. It's also unclear to me if they...

      According to Tesla's Vehicle Safety Report:

      Because every Tesla is connected, we’re able to use the billions of miles of real-world data from our global fleet – of which more than 9 billion have been driven with Autopilot engaged – to understand the different ways accidents happen.

      It's more than 9 billion miles by now, though I'm unsure of "when" is. The page was last updated in Jan 2023 at least.

      It's also unclear to me if they lump Autopilot and FSD together for the purposes of this reporting, but FSD hasn't been around as long. Additionally, the data Tesla presents here is biased for themselves, as systems like Autopilot are mostly used on highways, while the data the NHTSA is using is for all vehicles crashes.

      1 vote
  3. [2]
    disk
    Link
    I've been repeating this since day one, but a technology in such an undeveloped state should have no place in "real" environments. Nobody writes firmware for a 737 and tests it by flying a plane...

    I've been repeating this since day one, but a technology in such an undeveloped state should have no place in "real" environments. Nobody writes firmware for a 737 and tests it by flying a plane full of passengers to Paris. There is significant scrutiny when it comes to writing control system software, and with good reason.

    This stubbornness to avoid deploying LIDAR, whilst using your own customers as QA in what are potentially dangerous environments, show a clear lack of care the likes of which remind me of the Ford Pinto scandal.

    18 votes
    1. jcrash
      Link Parent
      Probably not a great example, as Boeing did do this on the 737 Max causing 2 fatal plane crashes. Your point however, stands.

      Nobody writes firmware for a 737 and tests it by flying a plane full of passengers to Paris.

      Probably not a great example, as Boeing did do this on the 737 Max causing 2 fatal plane crashes. Your point however, stands.

      27 votes
  4. [4]
    anxieT-rex
    Link
    This is the unfortunate point that we've reached in the name of capitalism and share value at the cost of everything else. As someone now in their late 30's everything feels so hopeless and any...

    This is the unfortunate point that we've reached in the name of capitalism and share value at the cost of everything else.

    As someone now in their late 30's everything feels so hopeless and any apparent progress feels fake or has major caveats, especially compared to how the world felt in the '90s.

    We need so much more regulation in basically every industry to combat the major issues like climate change, corruption, monopolization, income inequality, etc yet every day just seems to pull us further in the opposite direction.

    7 votes
    1. [3]
      andrewsw
      Link Parent
      Just wait another 20 years and you'll really feel it... mid-50s here and those feelings are even stronger than they were in my 30s. This idea, that (outside of medicine) products should be tested...

      Just wait another 20 years and you'll really feel it... mid-50s here and those feelings are even stronger than they were in my 30s.

      This idea, that (outside of medicine) products should be tested in the real world, for profits, until regulation catches up, has been around for a long time. Look at the chemical industry, the oil industry, really, any industry. Products are developed and released and the damage is left for future generations to deal with. In the older industries, it tends to be a little more slow moving -- waiting for a pesticide to build up enough to have a noticeable impact before any attempt at legislation comes around, for example. This lets the business get away with making profits and maybe never have to deal with the fallout.

      What we're missing, in my opinion, is the corporate death penalty. Bad behaving companies that externalize costs to the detriment of the people or the planet should have their charters revoked, their assets seized and liquidated, with the proceeds used to do the best clean that can be done. Investors can take a bath, boo hoo for them. Their investment performed poorly, oh well.

      It's all pretty demoralizing, though. It feels like we've crossed a line, gone through a one-way door, and we're never going to rein it all in.

      9 votes
      1. [2]
        supergauntlet
        Link Parent
        Corporate death penalty is an incredible idea, I don't know how I've never heard it mentioned before. What do you do with the leftover cash though? assuming you have any left over of course....

        Corporate death penalty is an incredible idea, I don't know how I've never heard it mentioned before. What do you do with the leftover cash though? assuming you have any left over of course. Donated, taken by the state, pay out extra severance to workers?

        3 votes
        1. andrewsw
          Link Parent
          Honestly don't know. I'd be surprised if there'd ever be money left over. But if there were, I'd say distribute it to the population at large as a sort of reparation for the damage caused.

          Honestly don't know. I'd be surprised if there'd ever be money left over.

          But if there were, I'd say distribute it to the population at large as a sort of reparation for the damage caused.

  5. Eji1700
    Link
    Many people have been saying this since day 1, and it really shows just how crummy regulations are that this was even allowed. Automation is a powerful tool, but we're decades away from something...

    Many people have been saying this since day 1, and it really shows just how crummy regulations are that this was even allowed. Automation is a powerful tool, but we're decades away from something like FSD and it'll probably never work under current conditions.

    6 votes
  6. elight
    Link
    MKBHD is right: as a Tesla owner, you're surprising the car to ensure that it doesn't do anything weird. It's not ideal. But, in my experience, it's far less stressful than engaging with traffic...

    MKBHD is right: as a Tesla owner, you're surprising the car to ensure that it doesn't do anything weird. It's not ideal. But, in my experience, it's far less stressful than engaging with traffic unmediated by Tesla Autopilot. 4 years, no accidents, 25k miles.

  7. [24]
    Comment removed by site admin
    Link
    1. [6]
      mynameisnotdoug
      Link Parent
      Musk didn't remove LIDAR. It never had LIDAR. They did remove RADAR in recent models.

      Musk didn't remove LIDAR. It never had LIDAR. They did remove RADAR in recent models.

      23 votes
      1. [5]
        TAn0n
        Link Parent
        Really strikes me as a difference without distinction.

        Really strikes me as a difference without distinction.

        10 votes
        1. [2]
          terr
          Link Parent
          I can't speak to any technical differences between the two, but at least as far as terminology goes why not strive to be correct?

          I can't speak to any technical differences between the two, but at least as far as terminology goes why not strive to be correct?

          16 votes
          1. TAn0n
            Link Parent
            I guess. The article itself conflates the data and completely misrepresents the results. The distinction of LIDAR/RADAR to the purposes of this discussion seems moot.

            I guess. The article itself conflates the data and completely misrepresents the results. The distinction of LIDAR/RADAR to the purposes of this discussion seems moot.

            2 votes
        2. Omnicrola
          Link Parent
          They are good at different things. Fundamentally they are both the same, they're doing distance calculations based on the reflected energy of radio waves (RADAR) and light waves (LIDAR). However...

          They are good at different things. Fundamentally they are both the same, they're doing distance calculations based on the reflected energy of radio waves (RADAR) and light waves (LIDAR). However because of the properties of those different EM wavelengths and how they interact with physical materials, they behave differently when using them for range finding. RADAR is better at longer ranges, whereas LIDAR is better at shorter ranges.

          For a more in-depth technical breakdown, I found this to pretty informative : https://flyguys.com/lidar-vs-radar/

          I can't really speak to the rationale around why Tesla removed it, but it does seem like a self-handicap to not have RADAR or LIDAR. Optical cameras and image processing also have their own strengths and weaknesses.

          11 votes
        3. mynameisnotdoug
          Link Parent
          Nothing wrong with being accurate. I’d argue explicitly excluding LIDAR from the get go is worse than cutting radar after the fact.

          Nothing wrong with being accurate.

          I’d argue explicitly excluding LIDAR from the get go is worse than cutting radar after the fact.

          8 votes
    2. [12]
      guamisc
      Link Parent
      There are plenty of criticisms to level at Tesla, but the argument makes such a basic attribution error comparing total fatalities with any assistance (all Teslas have some form of driver assist)...

      There are plenty of criticisms to level at Tesla, but the argument makes such a basic attribution error comparing total fatalities with any assistance (all Teslas have some form of driver assist) with solely FSD miles (only ~19% of Tesla's have FSD in the US) that it makes the whole argument just laughably bad.

      I wish we could get actual statistics out of Telsa, but the simple fact that the Tesla routinely scores higher than most other cars in vehicle safety tests. That performance would lead one to believe that the claim "Tesla’s FSD system is likely on the order of ten times more dangerous at driving than humans." as questionable at best. You'd have to assume that Teslas are involved in a combined 10x more dangerous combo of crash severity, crash frequency, and occupant injury and I'm not about to make that assumption. Nor should you.

      The militant anti-Tesla crowd makes these kinds of methodology errors all the time.

      Full disclosure, I own a Model Y. It's the most fun driving I've ever had, the safety systems/warnings are significantly better than any car I've ever driven before. I am never going back to an ICE car. I will probably also not buy another Tesla so long as Elon is involved heavily, I do not wish to give him any more dollars.

      18 votes
      1. [2]
        Minty
        Link Parent

        occupant injury
        Pretty sure it's more about pedestrians, cyclists, and other road users, not Tesla occupants.

        6 votes
        1. guamisc
          Link Parent
          That, to my understanding, is not the metrics quoted and used. Like I said, poor methodology and analysis.

          That, to my understanding, is not the metrics quoted and used. Like I said, poor methodology and analysis.

          2 votes
      2. [9]
        wisteria
        Link Parent
        Was looking for this. Programs still cannot keep up with humans and how unpredictable we can be. For example, what if a Tesla successfully stopped, but another car came swerving and collided into...

        Was looking for this.

        Programs still cannot keep up with humans and how unpredictable we can be. For example, what if a Tesla successfully stopped, but another car came swerving and collided into it?

        We need more data to understand the cause and nature of crashes—among other things—before immediately jumping to "Tesla causes more crashes!"

        1. [5]
          Akir
          Link Parent
          This is the thing that has always made me skeptical about automatic cars driving on the same roads as humans. Even ignoring the basic traffic violations that seemingly everyone does (changing...

          Programs still cannot keep up with humans and how unpredictable we can be.

          This is the thing that has always made me skeptical about automatic cars driving on the same roads as humans. Even ignoring the basic traffic violations that seemingly everyone does (changing lanes without signaling, speeding, etc.), I still see people doing stupid and unpredictable things every day.

          By all means, automatic driving systems should never work in car parking lots, where there are basically no rules or guidelines.

          3 votes
          1. [4]
            wisteria
            Link Parent
            Now that we do have automatic driving systems (ADS) that may very well be safer, it would make sense to regulate driving more (i.e., mandate ADS) to reduce drunk driving, reckless driving, etc....

            Now that we do have automatic driving systems (ADS) that may very well be safer, it would make sense to regulate driving more (i.e., mandate ADS) to reduce drunk driving, reckless driving, etc.

            But that might be controversial.

            1 vote
            1. Beowulf
              Link Parent
              I think the big issue here is that ADS is (I think, but may be wrong) more dangerous if a majority of drivers aren't using it. The lifespan of a car is very, very long. I regularly see cars that...

              I think the big issue here is that ADS is (I think, but may be wrong) more dangerous if a majority of drivers aren't using it. The lifespan of a car is very, very long. I regularly see cars that are 20+ years on the road. In fact, the average age of cars on the road gets older every year. So, even if you mandated that every car sold starting tomorrow had ADS, it would take 12+ years for 50% of cars on the road to have it.

              2 votes
            2. [2]
              xathien
              Link Parent
              I've often wondered what it would be like to have a lane of traffic on the freeway that was designated as a "computer-driven express lane", similar to current HOV express lanes. And then, as...

              I've often wondered what it would be like to have a lane of traffic on the freeway that was designated as a "computer-driven express lane", similar to current HOV express lanes. And then, as uptake increases, just slowly expand it across other lanes. This would ideally reduce meat-driven vehicles and increase consistency of driving.

              I'm sure this idea is far-fetched and infeasible in a thousand ways, but it makes me happy to imagine.

              2 votes
              1. wisteria
                Link Parent
                I wouldn't say it's far-fetched or infeasible, I would just expect people to try to use that lane to avoid traffic and whatnot. It's a solid idea, nonetheless.

                I wouldn't say it's far-fetched or infeasible, I would just expect people to try to use that lane to avoid traffic and whatnot.

                It's a solid idea, nonetheless.

                1 vote
        2. [3]
          guamisc
          Link Parent
          I mean Tesla's being in more crashes per driven miles would be incriminating even without the full context, but they're using the wrong numerator or denominator in their calculation. It either...

          I mean Tesla's being in more crashes per driven miles would be incriminating even without the full context, but they're using the wrong numerator or denominator in their calculation. It either needs to be FSD fatalities/FSD miles or AP+EAP+FSD fatalities/AP+EAP+FSD miles.

          They can't use the larger AP+EAP+FSD fatalities numerator and the much smaller (<10% of the larger figure's miles) FSD miles.

          1 vote
          1. [2]
            wisteria
            Link Parent
            I'm not going to pretend I know everything about this subject, but I'm still curious. I read @miyu's comment here about only 17 deaths since 2021 from Tesla US compared to 43,000. Not even...

            I'm not going to pretend I know everything about this subject, but I'm still curious.

            I read @miyu's comment here about only 17 deaths since 2021 from Tesla US compared to 43,000. Not even thinking of fatality rates—what's the rate of crashes from autopilot Teslas compared to rate of crashes from all other drivers? I think that information would give us a better idea of what's actually happening here.

            The fatality rate could be higher just because their build isn't protective enough, not because their autopilot is unreliable, right? Correct me if I'm wrong.

            1. guamisc
              Link Parent
              Teslas routinely rate as one of if not the highest safety fleet of cars that exist. In order to get a fatality rate 10x worse than average they'd have to be more than 10x as frequent and severe in...

              Teslas routinely rate as one of if not the highest safety fleet of cars that exist. In order to get a fatality rate 10x worse than average they'd have to be more than 10x as frequent and severe in accidents than the other cars in the US. Is that plausible? No. Extraordinary claims require extraordinary evidence.

              The supposition that the article makes about Tesla having a high fatality rate isn't supported by evidence and only appears in their calculated numbers because of an incorrect application of stats and absolutely horrific assumptions.

              1 vote
    3. [5]
      Captain_Wacky
      Link Parent
      If I was a NASA official, I'd be looking at SpaceX's work on Starship with the utmost scrutiny. Sure, Tesla and SpaceX are two entirely different entities, and sure the Falcon launches have been...

      If I was a NASA official, I'd be looking at SpaceX's work on Starship with the utmost scrutiny.

      Sure, Tesla and SpaceX are two entirely different entities, and sure the Falcon launches have been an overwhelming success, but it's also clear that the success of both companies have let Elon's ego go absolutely rampant.

      He would absolutely be dumb enough to feed NASA some doctored numbers.

      6 votes
      1. [3]
        CannibalisticApple
        Link Parent
        I'm just remembering how the recent launch failed and destroyed the launchpad. Launches failing isn't anything new, but to mess up the launchpad deserves some side-eye.

        I'm just remembering how the recent launch failed and destroyed the launchpad. Launches failing isn't anything new, but to mess up the launchpad deserves some side-eye.

        11 votes
        1. [2]
          AFuddyDuddy
          Link Parent
          IIRC that was a decision Musk himself, that the pad didn't need such a deep pit. Musk is not an engineer, but he pays the bills. Now he's going to pay bigger bills because he didn't listen to his...

          IIRC that was a decision Musk himself, that the pad didn't need such a deep pit.

          Musk is not an engineer, but he pays the bills.

          Now he's going to pay bigger bills because he didn't listen to his engineers

          4 votes
          1. mild_takes
            Link Parent
            I believe it lacked exhaust vents. I don't think that's the correct term, but the things that redirect exhaust to the side(s). Last I heard his plan was to build a water sprayer underneath the...

            I believe it lacked exhaust vents. I don't think that's the correct term, but the things that redirect exhaust to the side(s).

            Last I heard his plan was to build a water sprayer underneath the rocket that is a pair of metal plates where one side is perforated. You know... rather than just look at what NASA did 70 years ago.

            1 vote
      2. Minori
        Link Parent
        SpaceX isn't really run by Elon, and that's a large part of why it has been so successful. We can also compare SpaceX to basically every other rocket manufacturer around the world and see that...

        SpaceX isn't really run by Elon, and that's a large part of why it has been so successful. We can also compare SpaceX to basically every other rocket manufacturer around the world and see that they're wildly successful with their reusable designs.

        7 votes