Posted
by
timothy
on Thursday November 21, 2013 @04:15PM
from the all-the-kids-are-above-average dept.

cartechboy writes "There's always that kid in the class that ruins it for everyone when being graded on a curve. At the moment, that kid is Tesla and Elon Musk. Tesla's been proudly claiming the Model S is one of the safest cars in the word despite the recent fire controversy. And while it may be just that, claiming it earned 5.4 stars from NHTSA isn't pleasing the safety agency as there is no such thing as a rating higher than five. While NHTSA already released a statement indirectly to Tesla saying it doesn't release ratings higher than 5, Tesla continued to promote this fictitious rating. Now NHTSA has updated its guidelines explicitly stating safety ratings are whole numbers only and that 5 stars is the maximum advertisers can claim. If advertisers and automakers decide to disregard these rules NHTSA is threatening removal from the program or referral to state authorities for appropriate action. Basically, hey Tesla, stop making false claims."

NHTSA does not publish a star rating above 5, however safety levels better than 5 stars are captured in the overall Vehicle Safety Score (VSS) provided to manufacturers, where the Model S achieved a new combined record of 5.4 stars

I can understand the NHTSA complaint (Tesla's claims could be misleading out of context), but its not like Tesla is making any false claims.

Also, I recall this claim / story being about 3 months old at this point, and I believe NHTSA complained around the same time. Is slashdot seriously that far behind, or (as I suspect) is this an attempt to generate additional controversy and angst due to the other Tesla stories in the news?

Also, I recall this claim / story being about 3 months old at this point, and I believe NHTSA complained around the same time. Is slashdot seriously that far behind, or (as I suspect) is this an attempt to generate additional controversy and angst due to the other Tesla stories in the news?

Neither. (Well, it could be the latter.)

Rather, it is new action by NHTSA. "Complaining" is a lot different from saying "we will stop accrediting your cars". The former is old news. The latter is, well, news. (The "guidel

It reminds me of when the laboratories that rated sunscreens could only rate them up to SPF 15, and pretty much every sunblock on the market was exceeding that. At some point, at least in New Zealand and Australia, the government stepped in and said they couldn't advertise higher than 15, so they all became SPF15+ for a while.

Stay tuned for Elon Musk's critique of the NHTSA and long blog post detailing why the NHTSA is a bunch of corrupt scumbags and how Tesla is so awesome that it is able to get ratings above a perfect score.

Yeah, Musk is self-righteous and crazy intense about the PR for his companies. However, this seems like a warning that was justified, and at the same time might not have been foreseen by those making the claims, because it's kind of a technical quibble.

No, he's not correct. If he wants to go into more detail about safety than the NHTSA does, or make claims on his own, that's fine. He might (possibly) even know more about car safety than they do. What he's doing though, it pointing to the NHTSA as an authority that's made a specific claim. The NHTSA isn't willing to stand by that claim, however, so he's being misleading at best, and lying at worst.

No, he's not correct. If he wants to go into more detail about safety than the NHTSA does, or make claims on his own, that's fine.

They are correct, you are wrong.

"NHTSA does not publish a star rating above 5, however safety levels better than 5 stars are captured in the overall Vehicle Safety Score (VSS) provided to manufacturers, where the Model S achieved a new combined record of 5.4 stars."

The NHTSA is pissed because you're not supposed to TELL people what the ACTUAL safety rating is.

For example, the Tesla's VSS works out to 5.4 stars prior to rounding, another car has 4.5 stars prior to rounding. Despite being almost an entire "star" apart, both these vehicles get "5 stars" and appear to be just as safe to the consumer. The NHTSA is pissed because they don't want the public to be aware of it, because people will rightfully ignore the star ratings and demand th

About 80% of the comments already seem to be talk about how the NHTSA actually did give them a 5.4, but only allows them to advertise whole numbers and nothing above 5. So... it's a technical dispute over bureaucratic assholery.

To play devil's advocate for a second, measurements like the safety ratings inherently have error to them. For something like car safety, is a 5.4 really better than a 5.3, or was that just a quirk of the particular tests they did, and the 5.3 would be safer on the road?

Look at it from the NHTSA's perspective: if you think that Tesla's advertising is making claims that aren't particularly supportable because of margins of error like that and they're using your data to do it (and in the process saying essentially "NHTSA says we're the safest car on the road" when you don't want to make that claim), I think you'd be well within the realm of reasonableness to make them stop it.

To play devil's advocate for a second, measurements like the safety ratings inherently have error to them. For something like car safety, is a 5.4 really better than a 5.3, or was that just a quirk of the particular tests they did, and the 5.3 would be safer on the road?

Not really.

What happens is cars are rated to the current safety rating - the reason you can score above 5 is because the number is based on the raw figures and the current weightings.

The NHTSA records down in its database the raw numbers, then uses those numbers to calculate the safety rating based on the current weightings (from empirical data). This lets them recalculate the safety rating as need be - yesteryear's 5 stars may be this year's 3 stars, for example. Or, depending on how cars individually perform, it's possible two 5 star cars with the old rating may become a 3-star and a 4-star car.

So you cannot compare "stars" between model years, but you can compare them with historical vehicles recalculated to new standards. After all, many old 5 star vehicles may lack the safety features present on today's modern vehicles, so they won't be 5 stars anymore in the current rating.

The rating will go down as new model years and new tests are introduced - after all, we'd have hit 5 stars 50 years ago if the tests didn't change. The NHTSA updates its tests and ratings when too many cars are pegged - and there's a new test that apparently reflect the more common crashes that many "5 star" cars now fare poorly on.

Next year, the 2013 Tesla Model S may drop from 5.4 to 4.3. But the 2014 Tesla Model S may still get a full 5 stars because Tesla anticipated the new tests and built the cars to withstand them appropriately.

Musk doesnt need a rebuttal, since (if Im reading this right) NHTSA didnt contradict his claim that the internally provided NHTSA data showed a 5.4 rating; their objection appears to be that, for advertising / marketing purposes, the "official" NHTSA numbers end at 5 and you arent supposed to quote NHTSA as assessing a higher number than that.

Maybe Im mis-reading this, but the Tesla press release from August even said as much-- that the "public" rating was 5 stars, but the "eyes only" manufacturer assessme

After that whole "My new supermegatube will do everything your high speed railway does, only cheaper - restrictions apply, supermegatube does not, actually, go to any of the locations served by high speed rail, and actually isn't going to be built anywhere near 50% of the stops" crap from Musk, I do feel the guy isn't always speaking with his honesty valve sealed properly.

I don't know why this was modded as funny, it's right out of the article:

NHTSA does not publish a star rating above 5, however safety levels better than 5 stars are captured in the overall Vehicle Safety Score (VSS) provided to manufacturers, where the Model S achieved a new combined record of 5.4 stars.

There are lots of cars with five star crash ratings - for all we know, some of those have received a 5.2 or 5.3 or 5.39 or whatever on that particular test. But those other cars' manufacturers have always used the scoring system as intended by the NHTSA, and just listed "5".

Musk is basically trying to make it look like there's a safety gap between his car and all the other five-star-rated cars, when there's no evidence that gap actually exists.

Which cars don't have a 5 star crash rating? What's the lowest rating for being allowed on the road? Seems like the ratings are kind of useless if every car is rated with a 5. Just from clicking around on their site, I randomly selected 5-6 cars, and all of them had a 5-star overall crash rating.

There are still some cars that get 4-stars, but this particular model (RAV4) got several 4-star ratings, prompting newspaper articles about "failing" safety tests. People clearly expect perfect security and safety all the time at all costs. (See: Patriot Act)

I think it's important to consider the purpose of the grading, which is not to differentiate cars based on minor if statistically significant differences, but to really demonstrate which cars are objectively less safe.

It's not on a curve, it's an absolute scale. A variable scale would effectively leave carmakers unsure of what regulatory bar they had to meet. Imagine a carmaker that produced an incredibly safe, unaffordable fortress that wrecked the curve. I'm very glad that a system such as that has encour

Plenty. Didn't last year's Toyota Camry come in at 3 stars? I don't see anything below that too often though.

Of note though is the Chevrolet Aveo in 2004 received 4 stars from the NHTSA and yet the same car in Australia (Holden Barina) received 2 stars from the Monash University's Accident Assessment Program. Kind of makes you wonder what the point is.

Yet that's the actual, accurate score. I don't see why the actual score can't be reported?

There is no analytical basis to explain exactly how 5.1 is less safe than 5.4 and the analysis makes no such claim. If the NHTSA allowed manufacturers to abuse the figures by claiming these fractions are meaningful then the rating system would lose credibility. Ultimately manufacturers might game the system to amplify a meaningless fraction.

Tesla had this explained to them and Tesla ignored it. Now the NHTSA has had to get official on their asses and tell them to stop. This is Tesla's own fault, whether the fanbois like it or not.

There is no analytical basis to explain exactly how 5.1 is less safe than 5.4 and the analysis makes no such claim. If the NHTSA allowed manufacturers to abuse the figures by claiming these fractions are meaningful then the rating system would lose credibility.

If that's true, then there's also no analytical basis to explain how exactly 4.4 (Which you rounded down to 4) is less safe than 4.6 (Which you rounded up to 5).

The fact of the matter is, if such is your degree of error, then You have to make distinctions for which there is no analytical basis;At some point, if you are truncating the number: you have to choose which whole number you will truncate it too, AND another figure that is only 0.1 different, will appear as a whole POINT less safe or more safe than the other --- while other numbers that differ by 0.4 or more, will be truncated to the same value.

Listing the more detailed figure is not per-se an abuse, then.

There is false precision there; and there is also false precision truncating the number+fractional part to one of the surrounding whole numbers.

Conveying the decimal point conveys false precision, BUT at the advantage of eliminating bias and haphazard truncation.

You can actually see that one car got 4.4, and another got 4.6; instead of seeing "4 and 5"; Off by a whole star ------ there is something to hand you a relative showing, that they're really rather close, maybe the same.

There is false precision there; and there is also false precision truncating the number+fractional part to one of the surrounding whole numbers.

Conveying the decimal point conveys false precision, BUT at the advantage of eliminating bias and haphazard truncation.

You can actually see that one car got 4.4, and another got 4.6; instead of seeing "4 and 5"; Off by a whole star ------ there is something to hand you a relative showing, that they're really rather close, maybe the same.

The problem isn't false precision. It's not even false accuracy (which is what I think you meant). It is the conceptual basis on which the safety scores are based. The NHTSA specifies certain criteria and example accident scenarios and then computes a score. This score only strictly speaking applies to the particular accident types covered by the testing. Clearly the testing is going to be reasonably representative, but it will not be possible to design a test suite that is fully representative of all possi

The NHTSA doesn't want manufacturers to optimise for the particular suite of sample accident scenarios to gain an extra 0.1 score and beat their rivals, because that would not mean that the real-world safety was improved and might even mean that safety declined slightly in non-tested accident scenarios. By rounding the scores it eliminates the motivation for this pointless effort,

If that were true, they wouldn't give the precise VSS scores to the manufacturers, just a "where you failed" review for individual tests. In the scenario Mysidia described (4.4 versus 4.6), there's every reason for a manufacturer to want to sneak above that.5 mark, in order to get a "NHTSA 5-Star Rating! * * * * *". If you were a manufacturer, wouldn't you give up a few percent of real-world safety for that extra star (by sneaking from 4.4 to 4.6)? Say by enlarging the rear pillars to cheaply improve your inverted-drop roof-crush score (your area of lowest performance) in spite of loss of visibility it causes (and thus increased real-world accidents), rather than actually improving roof-crush performance through proper structural changes. Or sacrificing your top score in roll-over stability by adding or removing elements to improve your very low-scoring rear collision survival test. Whereas if you weren't given a VSS score for each test, you would only know that the review says your rear-collision survivability sucks, and you'd have to work to improve it without reducing safety in other areas (because you wouldn't know where you exceeded the tests and thus have margin to sacrifice.) The current system clearly encourages manufacturers to "build to the test" at the expense of real-world safety.

That is totally misleading. Hell, it's a lie!! Tesla just notified me that I can no longer drive my car. I've been told I will have the option to buy an extremely similar car for 5 times the price, though. Can you confirm?

Elon Musk would never have accomplished what he has if it wasn't for his rabid passion and forceful, egotistical personality. He is trying to change the status quo in an industry where many have tried and failed, and where many want Tesla to fail as well. He had best try to temper himself, though; lest he become our generation's Nikola Tesla/Howard Hughes in more ways than one.....

I am glad he has fire and passion. He is fighting against a lobby that spans centuries, so even if his claims might be a bit over the top, the people who want to seem him out of business are many, and are THE richest people out there.

He is competing against the most powerful people on this planet, so it is pretty darn amazing he has done this well. He also has showed the auto industry is stagnant -- his first commercial vehicle faster than most sports cars except the high end Italian makes? Impressive. Same car using a completely new drivetrain? More impressive. Same car with zero deaths? Still more impressive.

Elon Musk may be a bit of a blowhard, but he is actually effecting change.

Compared to Teslas, what other electric vehicles out there can smoke even an average sports car like a Corvette? A Leaf? Maybe in free fall. A Prius. Nope. Elon Musk has made a completely new car category, something not seen since the soccer moms wanted station wagons back, but didn't want them called station wagons, so they were named crossovers.

During Tesla's initial work on AC power, electricity was a luxury for the rich rather than the common man's utility. And the "particle beam weapons, remote control devices, and wireless power" didn't exactly get to "majority of the populace" status, either. Giving Tesla credit for every engineering advancement in those fields over half a century since his death is a bit excessive hero-worship. Tesla did a lot of great stuff, but a lot wasn't particularly useful at the time (or even later, for the more wildl

If safety is a good thing, and it's a good thing for consumers to prefer safer cars, why not add a decimal point or two?If two cars are equally qualified in the minds a specific customer - and one car is 0.1 stars higher, why let that car get more sales?

Dude, let the free market drive safety higher! Just like EPA window stickers. Give the customer information. If Tesla really did rank 5.4, then let the other manufactures get some public shaming. Maybe they can respond to purchasing competition & catch

If Tesla really did rank 5.4, then let the other manufactures get some public shaming. Maybe they can respond to purchasing competition & catch up and make their cars safer.

Again - since Musk is the only one who hasn't played by the rules here, why are you assuming the other five-star-rated cars are exactly 5.0? There likely have been others that scored 5.2, 5.3, or 5.39 on that particular sub-test - but their manufacturers followed NHTSA guidelines when promoting their vehicles' safety.

You really have no evidence that the Tesla tested out as being significantly safer than those other five-star cars.

Apparently people aren't reading what's been said. Tesla's press release says: "National Highway Traffic Safety Administration (NHTSA) has awarded the Tesla Model S a 5-star safety rating", and "NHTSA does not publish a star rating above 5". Thus Tesla is not claiming that they were assigned a 5.4 since they outright acknowledge that NHTSA doesn't publish a rating above 5. What Tesla did say is that if one were to take the individual scores that were provided by the NHTSA (which apparently includes ratings above 5, and possibly decimal as well) and average those, the resultant number would be a 5.4.
Now what is probably getting the other manufacturers upset is that the clipping of the results at 5 means that the vehicles that just squeaked into the 5 look the same as vehicles which may have blown past the 5. If they didn't like that, why aren't the individual scores also integral and clipped at 5? Then one could not possibly claim (or even appear to claim) a number higher than 5.
So, this whole release is trying to beat up Tesla for something they didn't say. They didn't say that the NHTSA awarded them a 5.4 rating (see the first quote). They did say: "achieved a new combined record of 5.4 stars.".

Here in the USA, the grid is 68% fossil fuels [eia.gov]. So unless Tesla is including a free ZPM [wikia.com] with every purchase, "Zero Emissions" is a crock of shit, just like a 5.4 star safety rating.

Huh? 180% efficient by getting 1.8 times the energy out than you put in? Our fuel problems are solved! Just have a few of these 180% efficient cars sitting there and charge up the first one, then charge two cars to 90% from that. Next charge two other cars to 100% from the 90% cars, leaving you with two fully charged cars and two with about half charges. Roll up two more cars and charge them from the half charged cars... Now you have three fully charged cars from just one... See where this is going?

If you charge it with carbon-sourced electricity, the Tesla, a vehicle, still doesn't generate emissions.
If you strap a diesel generator to the roof to run your personal electronics, the Tesla, a vehicle, still doesn't generate emissions.
If you load up the trunk with flatulent cattle, the Tesla, a vehicle, still doesn't generate emissions.

That some people have trouble parsing natural language is not Elon's problem.

We're not all interconnected. I was one of the people, during Y2K planning, who made dang sure of that, so we wouldn't get a cascade grid failure if any of the circuitry software glitched. There are reasons for that - we literally had people at the physical switches just in case.

I get that you hate America and that Texas is going to wind power and solar power, but back when I was born in Texas it was a Blue state too.

I often see other cars with Partial Zero Emissions stickers. This confuses me as any part of zero is zero.

you are mathematically correct. It's bureaucratic gobbledygook.

The PZEV rating [wikipedia.org] basically means that the car's emissions of everything *other than* CO2 are about as close to zero as you can get within the limits of chemistry. So the catalytic converters are top notch, the fuel handling system allows for ~0 vapor loss, NOx and CO out the tailpipe are nearly zero, particulates are avoided or capture

I often see other cars with Partial Zero Emissions stickers. This confuses me as any part of zero is zero.

That's because you have to think like a california bureaucrat to understand this. It's not partially zero, it just partially counts towards meeting a zero-emission requirement.

In 1990, california passed a law that 10% of the cars for sale in california by 2003 needed to be ZEV zero emission vehicles. Note that ZEV doesn't mean zero-emissions for the environment, but simply zero tail-pipe emissions (e.g., electric cars or hydrogen fuel cells). The California Air Resources Board (aka CARB) that was tasked t

Yep. The NHTSA apparently gives the manufacturers the individual ratings in each category, which presumably go above 5 and may be decimal. They may have some sort of agreement that they aren't allowed to publish the individual ratings.

No, you don't get it.All volume controls go to 10.Nigel's go to 11. They are one louder.Similarly, all safety ratings go to 5.Tesla's goes to 5.4, it is.4 safer. For example say that you are in the safest car you could find, it is all the way up to 5 stars safety rating. All the way up. But you want it a little more safer, to be able to go over the cliff, so what can you do? That's where Tesla comes, giving you that extra.4.

The claim is false: The NHTSA rates cars on a 5-point scale, and gave the Tesla S a 5-point rating, the highest they could get. This rating is based on several sub-ratings, where the Tesla also got 5-point ratings, in all categories.

Tesla is basically trying to claim for marketing purposes the fact that they got 5-point ratings in all of the subcategories (which isn't necessary for a 5-point overall rating, and in fact is extremely unusual, if not unique) means that they got 'better than a normal 5-point r

As far as I can tell, Tesla claims - as do several news outlets - that the NHTSA also releases some other raw numbers to the manufacturers which Tesla then decided to 'combine' (whether that's adding or averaging or whatever - who knows.) to give a 5.4 .

Really, the issue is lack of transparency - since we, the public, don't get to see those numbers. Thus we can't really give a good opinion other than "NHTSA says 5 is the maximum. THE MAXIMUM!" and all nod in agreement at the overlord's words apparently for fear of getting booted out.

So it had the best overall test score.. but is not necessarily the safest. But the test is on safety. So it's the best in safety.. but not necessarily the best in safety.

Maybe while they're quibbling they could come up with a system that makes sense to themselves, the manufacturers and, most importantly, the public. If in the end that means Tesla does get a 5.4 and they want to hang on to 5 stars - well I guess they'll just have to lower the rating on a bunch of other cars.

No, thats NOT what tesla claimed, their actual claim was exactly this:

NHTSA does not publish a star rating above 5, however safety levels better than 5 stars are captured in the overall Vehicle Safety Score (VSS) provided to manufacturers, where the Model S achieved a new combined record of 5.4 stars.

They said the highest rating you can advertise is 5. Tesla is saying that they were provided an internal, manufacturer-only rating that is higher than the publicly available "5 star" rating. NHTSA does not seem to contradict that.

You know what? We did vote for it. Let's see how it turns out rather than making wild predictions. If it fails, we own it. If it doesn't fail, we own it. The people paying you seem to be willing to pay any price in blood or money to make sure the Democrats fail. Why are they trying so hard to get rid of the law before it has a chance to work? Why not let it do its thing, and then when fire and brimstone rain from the sky, they can blame

Any paranoid with a lick of sense will tell you, the media is owned by Jewish bankers, not oil companies.

If the range issues scare people, they build charging stations (big oil is fighting the legality of these currently)

Uh, citation needed. Desperately.

If they have too good of a crash score, the score rating system is changed.

The system is the same as it's always been, there is no victim here (except consumers who don't know that the NHTSA rating system has always been a whole-numbers type that ranges from 0 - 5. OH, and people who don't realize that pretty much every new car on the road has a 4-5 star crash rating).

They are banned from selling in Texas because that's big oil's home turf....

There's no such thing as a "big oil" company any more - the big guys are all "energy" companies and care as much about natural gas. In places where electricity comes from natural gas, the energy companies may make more per mile from the Tesla than a gas-burner (maybe not this year, since natural gas is so cheap right now, but the logistics from oil drill to gas pump are complex and expensive, compared to selling natural gas to power companies).