Should Automakers Be Selling "Beta" Cars?

It’s Already Been Happening to Varying Degrees

In the introduction to a January 2009 BusinessWeek excerpt from his book “What Would Google Do?,” author Jeff Jarvis extolled the idea that the Detroit automakers should start releasing cars in “beta” form, much the way the Silicon Valley tech giant had been doing with its software products. Despite the obvious dangers of open public beta testing in the car business, it seems that a number of automakers, including Tesla, Fisker, BMW, Ford, General Motors, have been doing exactly that to varying degrees over the past several years.

One of the mantras of Silicon Valley technology companies is “Ship Early and Ship Often.” The idea is that as soon as you have a remotely viable product, you get it into the hands of users and then rapidly iterate to make the product better. For a small shop with a handful of developers producing a smartphone app, or even a larger team producing a Web application like GMail, that concept works great. In the capital-intensive world of building complex physical products like cars, that process can be a very costly kiss of death. The question is, should automakers use paying customers as beta testers without making it clear what they are doing?The history of beta testingThe concept of structured alpha and beta testing of software has its origins with IBM. Back in the 1950s, the computing pioneer defined alpha testing as the internal product verification before a public announcement. Beta testing was the verification of the product by users before it got released to production.Until Google went all-in on publicly releasing software that was labeled as beta in the past decade, beta releases were mostly restricted to select groups of testers who committed to providing regular feedback about problems in the product. Google’s GMail was the stereotypical example of an open beta, retaining that status for more than five years from its initial release in April 2004 until July 2009.In the automotive realm, with a few notable exceptions, manufacturers have generally kept their equivalent of beta testing strictly in-house, using engineers and test drivers to evaluate new vehicles and systems. Even the final stage of evaluation before vehicles go on sale to the general public has been restricted to employees. For example, at General Motors, anywhere from several dozen to several hundred production-representative examples are distributed to managers and other employees in the final months before the on-sale date in what is known as the captured test fleet.

Public beta testing in the auto industryPerhaps the best-known early example of a public beta test in the car industry was the Chrysler Turbine car program. Between 1963 and 1965, Chrysler built 55 turbine-powered coupes and loaned them to more than 200 drivers around the United States for three months at a time to gather real-world data and driving impressions.By the end of that decade, politicians and lawyers had taken direct aim at the auto industry. Within a few years, the American auto industry had become one of the most highly regulated industries in the world, and the screws have only gotten tighter since. The combination of general product liability claims with safety, emissions, and fuel economy standards from federal and California regulators has made it increasingly difficult just to conduct the basic business of building and selling automobiles. Doing anything as daft as loaning or selling experimental cars to ordinary consumers was seemingly out of the question.Enter the California Zero-Emissions-Vehicle mandateThen in 1990 along came the California Air Resources Board (CARB) with a new rule mandating that automakers start building zero-emissions vehicles (ZEVs). By 1998, at least 2 percent of the vehicles from the seven largest automakers in the state had to be ZEVs, a fraction that was to increase annually to 10 percent by 2003. The major automakers immediately began two parallel efforts to develop battery-powered vehicles they could sell while at the same time lobbying to kill the mandate.

By 1996, General Motors had its all-electric powered EV1 ready, as did other manufacturers. And it also looked increasingly likely that the industry would succeed in eliminating the mandate. As a result, most of the automakers that prepared EVs -- including GM and Honda -- opted to only lease rather than sell these first-generation battery-powered vehicles.With the demise of the original ZEV rule, GM opted to take back the 1100-plus EV1s it had leased and sent most of them to the crusher, arguing that the bespoke plug-in shared almost nothing with any other GM vehicle, and inspiring the film “Who Killed the Electric Car?” GM gathered a great deal of data during the five years the EV1s on the road that would later be put to use in both the fuel cell and Volt programs. For all intents and purposes, the EV1 ended up being a public beta test much like the Chrysler Turbine program.Unlike the Turbine program, which remains dead nearly five decades later, electric propulsion never entirely went away. When it became clear that late 1990s-era battery technology wasn’t up to the task for a mainstream audience, automakers turned their attention to hydrogen fuel cells. Along with the usual internal testing of fuel-cell vehicles, several manufacturers, including Ford, Hyundai, Honda, and Nissan, began loaning out hydrogen-fueled cars to government agencies, universities, and other organizations to gather real-world data. Honda even went so far as to lease one of its first-generation FCX cars to a California family in 2005.

GM, on the other hand, made plans for the biggest public beta test since the Chrysler Turbine trials. General Motors R&D built a fleet of more than 100 electric drive Chevrolet Equinoxes with electricity provided by hydrogen fuel cells. In late 2007, GM began loaning these vehicles free of charge to ordinary drivers in California, Washington, D.C., and New York for three months at a time as part of Project Driveway. Over the next three years, the fleet accumulated more than 2 million real-world miles.At around the same, Honda introduced its first completely bespoke fuel-cell vehicle, the FCX Clarity, and began leasing them to customers in 2009 for three-year terms. Only a few dozen of these cars are on the road in U.S., but Honda remains publicly committed to the technology.To pay or not to pay?In some respects, the GM and Honda fuel-cell programs were very similar in that they put “ordinary” American drivers behind the wheels of some of the most technologically advanced vehicles in the world and allowed them to use them however they pleased for months or years at a time. The distinction is that Honda is charging its users a $600-a-month lease and officially calls the Clarity a regular series production vehicle.Since these fuel-cell test programs launched, BMW has followed suit with two waves of test programs for battery-electric vehicles. The first wave began in 2009 and put more than 500 battery-powered Mini Coopers into the hands of drivers in America and Europe for one-year terms. That was followed 18 months later by a second-generation EV prototype based on the 1 Series coupe, dubbed the Active E. While BMW charged participants $850 per month to lease the vehicles, they were clearly identified as limited-time test programs.Like traditional tech industry beta programs, each of these manufacturers is collecting usage data from these vehicles and subjecting participants to regular debriefings about their experiences.

Shipping beta products as production-readyAs long as manufacturers are completely transparent about the fact that what they are delivering is not yet ready for prime-time, inviting the public to drive experimental vehicles is probably a really good idea. No matter how much planning engineers do, they can never anticipate and test every possible real-world scenario. Even the most conventional vehicles still display some hardware and software bugs when they first go on sale. For example, the 2013 Ford Escape saw five recalls in the first eight months after going on sale, four of which involved the 1.6-liter engine.As automakers have transformed their products into something that often seems more akin to consumer electronics in the last few years, customers have had to come to terms that they could be treated like beta testers. Among the more prominent examples of this were the early adopters of BMW’s iDrive more than a decade ago and more recently those who opted for the MyFord Touch system in their Ford and Lincoln vehicles.

As cars have integrated more technology features and drivers have demanded the ability to use mobile devices on the road, automakers have scrambled to devise new human-machine interfaces. The more basic systems that just added auxiliary audio or USB inputs to allow music playback and hands-free calls have been relatively trouble-free. However, those that incorporate either control knobs or touch interfaces along with voice recognition to access computer-like menus have caused some customers grief. Ford has been pounded mercilessly in quality surveys for the past two years because of the complex and slow interface in MyFord Touch along with its propensity to crash and reboot, a process that can take several minutes. Since the debut of the similar CUE system in Cadillacs in 2012, GM’s luxury brand has been getting similar complaints. Both Ford and Cadillac are scrambling to develop software updates to improve the performance and reliability of these systems.Among the newer entrants in the industry, Fisker and Tesla seem to be going even further in using their customers as beta testers and information conduits. One early Fisker customer posted extended video reviews of his new ride, including a 41-minute opus that focused just on the infotainment system. http://youtu.be/rK8MqjagHu8.Similarly, Tesla has had issues that seemingly should have been ironed out prior to production with the Model S. Posts on the Tesla Motors Club forum have complained about issues ranging from problems with the massive 17-inch touch-screen center console to the door latching system. Because Tesla has opted to incorporate virtually all of the usual center console controls into the touch display, having it refuse to wake up is even more of an inconvenience than glitches experienced by Cadillac and Ford owners.More troubling are the issues with the doors. Tesla has opted to use electric door handles that sit flush with the skin and are supposed to pop out when touched. Some customers are reporting that they frequently don’t work properly, so either the doors won’t open or they open randomly while driving. Fisker and Tesla have shipped numerous software upgrades, but the cars are still far from perfect.To beta or not?During and after the 2008 financial meltdown and subsequent bankruptcies at Chrysler and General Motors, many pundits from the technology industry claimed that the auto industry should be more like Apple and Google, taking more risks and being more innovative. Many automakers are now doing exactly that with varying degrees of success or failure.

What those tech luminaries often fail to recognize is that the auto industry is more like the aerospace industry than Silicon Valley. The American auto industry is among the most heavily regulated businesses in the world, and even when automakers meet the letter of every law, liability lawyers are still happy to pounce at the slightest hint of a problem. Automotive engineers have to keep in mind that while a crashing smartphone is annoying, a software or hardware bug at 70 mph can be lethal.Companies that produce nothing but software seem to be able to get by just slapping a restrictive end-user license agreement on the launch screen that absolves them of any responsibility for what happens with their products. Even computer makers such as Apple or Dell are selling products that cost from a few hundred to a few thousand dollars and are not expected to last more than a few years. Car buyers expect a car to run relatively reliably for a decade or more, and even the cheapest cars can last more than 20 years. The expectations are completely different.When automakers make it clear that they are running a test program like Project Driveway or BMW’s EV evaluations, customers can go in with their eyes open and know that they might experience some problems. Even if customers pay for the privilege of participating, they probably won’t mind, and the engineers can learn a lot.On the other hand, when paying tens of thousands of dollars for a regular production vehicle, mass-market customers expect it to work from the start, whether they are buying from startups like Tesla and Fisker or century-old stalwarts like Ford, GM, or BMW. Every one of these companies regularly market their new technologies as features that will make customers’ lives better, but sometimes they don’t.Perhaps the solution is for auto manufacturers to develop ways to do more real-world beta testing of new technologies so they can find out what doesn’t work before they commit to production, making sure everyone involved knows exactly what to expect.

The Big Three beta tested their products all through the 1970s and 1980s. Remember the Chevy Citation, Plymouth Volare, Ford Pinto and the Chevy Vega? We all know how that strategy turned out: an unending erosion of market share to Japanese Corollas and Civics, that were actually sold as finished products. Turned off an entire generation of US car buyers.

GM has always Alpha and Beta tested its' Corvettes. I've had a couple. Vette folks have learned never to buy one the first year offered. I submit: '53,'63, '68, '84, 90ZR-1, '98, '09ZR1 etc.

I couldn't help myself and bought a '90ZR-1 but included GM's longest,most inclusive,no-deductible warranty (pricey!) to CYA. Had to use it 23 times!! Invaluable Test info was gleaned by the Corvette (Re-)Action Center :-(

2014 C7 Stingray? If you're made out of money, pay up to be an Official GM Alpha Tester this Sept. If you are REALLY froggy wait til 2015 or so, and BETA Test ZR1 or GS or Z06 variants.

GM thanks you for your unpaid internship and they'll jump right on your cogent,first-person results.

It sometimes works. Obviously, if Tesla hadn't found out that their driving dynamics and touchscreens didn't work well in their beta cars and fixed the problem, the Model S would've been a complete failure, with no chance of COTY or any other attention it garnered. If Honda beta tested the new Civic, they probably wouldn't have gotten into the mess that they did. Obviously, I don't think these "beta cars" should be sold, just like beta testing video games usually comes at no financial cost to the gamers, but rather driven by randomly selected drivers found on the street. Even the most brilliant designers and engineers slip up. They're human. Sometimes it takes a look from someone else (in this case, the customers that they're trying to please) to spot a few mistakes, even serious ones.

The risks of beta testing cars are way to high to make it worth while. The risks definitely out way the rewards on this one. I personally would not advise driving a beta car, at least for not any considerable length of time. It could be fun to just take it out for a test drive though. Just stick to buying the cars we all know and love for now, and given the right amount of time and research then these cars will be on the market too.

As an automaker, there's no way I'd want beta hardware in the hands of people. And for that matter, I'm DEEPLY skeptical of beta software in critical systems. Who would want to own, or RESELL a car with beta software governing the throttle mapping, brake control, or steering systems? Or what about the software that governs detonation and anti knock? Beta testing sounds cool, until you consider the liability you expose yourself to. When a car's beta software fails, PEOPLE COULD DIE AND THEIR FAMILY WILL SUE YOU. That doesn't really happen with Google maps.

I feel like this is already happening electronically... and has been for some time. My 2004 Jag X-Type was "supposed" to get an update to allow voice recognition/control. While Jag screwed the pooch and went back on that, I feel like systems like MyFord/MyLincoln Touch were ultimately released in beta form.

Here's the issue. I don't think automakers either HAVE a good grasp on what they will be able to fix with updates moving forward, or (more likely) they tend to make product development decisions based on best case scenarios and cheapest decision. While you can update software moving forward, they are unlikely to offer updates to hardware if, by the time the REAL release is ready, they find out that the installed hardware is inadequate. I feel like this is partly the case with Ford products at the moment. As speed has been noted to be such an issue with MyFord/MyLincoln Touch products, there is only so much optimization of software that can be done to speed the product. If the hardware is insufficient then there's nothing you can do.

As long as the consumer KNOWS he is part of a beta test program; one would hope the price of the vehicle would reflect the assistance/participation of the buyer. Heck, if someone wants me to pay to test the first anti-gravity sports car, I would pay to play!

Not only "should" they, we should encourage it. Obviously products need to be developed to a certain point, but we should encourage automakers to offer new advances in technology and design. Otherwise we'll all be driving boring Camrys, while we wait forever while the kinks get worked out with the good stuff.

Since the vast majority of the population is a bunch of mindless sheep who is too stupid to realize that they're being used as beta testers, I doubt anything will ever come about as a rise up against the car makers. BMW isn't the only one who has released new technology to the public without testing it somewhere; another example is Honda with their SH differential systems being released to the public in their "test" mules. It will continue to happen and the consumers will continue to buy the new flashy crap that gets put out there.

BMW has been testing new technologies on the consumer for years without EVER acknowledging said advances were " not quite worked out".....the first year of every 7 Series introduction seems religiously plaqued with gremlins - going as far back at 1988.

As far as I'm concerned, the Tesla S feels more like a software product than an automotive one, considering the way bugs are periodically "fixed" through patchs remotely downloaded to the car.

So, "beta" test ? Sure, why not ? But small number and for free. Don't "sell" the car for almost $100,000 like a finished product.

But, hey, it looks like early adopters of the Tesla S are like technology fanboys willing to put up with all the kinks to be among the first to enjoy an innovative product, and still pay dearly for it. So I guess it's a win-win situation for them and Tesla.

And people, stop comparing early recalls for mechanical issues, which happen all the time on cars, with what happens with Tesla S cars: pushing patches every few weeks or so, and unreliable ones at that that create more bugs to fix while fixing previous issues is a software business model.

When your brand new car gets recalled for a defective hose, when it's fixed, it's fixed. Replacing the hose typically does not create more defective parts elsewhere in the car.

In a way, a car's first model year is a beta. The adage that you shouldn't buy a car in its first model year before they've had a chance to work out the kinks may not be as relevant as it used to be, but it's still something to keep in mind.

@mynameisj I agree with you completely... however this is becoming less and less the case. The last three pieces of software I beta tested netted me no reduction in the released price of the software. My "reward" was the privilege of getting to use it ahead of time.

@DamsCal "As far as I'm concerned, the Tesla S feels more like a software product than an automotive one, considering the way bugs are periodically "fixed" through patchs remotely downloaded to the car."

but for me I should be selling cars, even they are beta or not, and the Model S is both automotive and a software product. Look at the Volt, the software must be updated using a DVD ROM bought straight from the dealer, or bought online.

How are you determining the success of the two cars made by Tesla? If its by what the media is telling you is successful than I would hope you would expand your net off information on what you think determines a successful vehicle. Bottom line is that hybrid technology using battery or any form of nickel hydride technology is far more expensive than fesable to produce a vehicle with reasonable performance and minimal fossil fuel consumption at a price that most can afford not to mention make appealing to a more realistic group of people who can;t and do not want to purchase a six figure hybrid.