Posted
by
Soulskill
on Friday November 19, 2010 @04:07PM
from the computer-may-or-may-not-generate-black-holes dept.

Geoffrey.landis writes "According to an article in the New York Times, documents revealed in a lawsuit against Dell show that the computer maker hid the extent of possible damages due to a faulty capacitor in the computers it shipped from 2003 to 2005. Dell employees were told, 'Don't bring this to customer's attention proactively,' and 'emphasize uncertainty.' (PDF) 'As it tried to deal with the mounting issues, Dell began ranking customers by importance, putting first those who might move their accounts to another PC maker, followed by those who might curtail sales and giving the lowest priority to those who were bothered but still willing to stick with Dell.' In other words, the most loyal customers got the worst treatment."

Calling any company is becoming a contact sport. I was informed by Verizon that my credit card number they had was expiring. So go to Verizon and try to log in...no dice. They eliminated my login I've had for a few years. I recently got FIOS. So now I was invited to register. Hmmm...what's this? I must run Verizon's special piece of CrapWare on my Mac just to register for an account? No fucking way am I letting Verizon run anything on my machine. AT&T got a note from my credit card company and updated automatically...one more fucking game Verizon throws at your head.

What to do? Call Verizon...find all their numbers for this, that, and the other are connected up to the same damn phone bot. Try to navigate the phone menus, finally find a human who will hopefully take my new experi date . "Oh, that is a problem for our billing dept. Let me transfer you." "Uh, okay"....damn...back in phone menu hell right up at the top where I started. After 4 rounds of this, always reaching a different piece of proto-simian meat, I finally found a way to get to billing which seemed to surprise the billing person out of her nap. Finally fixed it. "Is there anything else I can do for you?" "No...NO NO NO...don't even think about it."

Along the way, I was offered free movies for my FIOS for three months...I was "eligible". No way, I've been through that before with Verizon and FIOS. 3 phone calls later I finally got them to drop the damn movies after they started charging me, hoping to slip it onto the bill without my notice.

"Let's bill your hardware charge to an account we'll close on you in 2 weeks from our side and then send it to a collection agency who sits on it for 4 months". That takes 4 hours to fix with your described Turbo Transfers. "Let me get you to billing. - No, we only handle Pennsylvania, let me get you your area - Oh, I am only billing, I can't take your credit card - I have no idea what that charge is, let me transfer you -..."

Then they are just barely able to install a dry-loop DSL with 11 phone calls over 3 weeks.

The only thing is, the reputation of Comcast scares me more so I haven't yet switched.

The worst comcast ever did was start billing me for a modem rental, then demand I return the modem I owned to stop. Sicced the PUC on them and it stopped real fast. Aside from that, the actual phone monkeys are decent, and I pay by check.

Of course, this can backfire. If you "emphasize the certainty" of not doing any more business with me, why exactly should I bother fixing your problem? Either way, you're not my customer anymore. Time helping you would be better spent on people who only "might" not do any more business with me.

Undoubtedly, but given that I've already lost you and you've stated with "certainty" that I've lost you, I may as well cut my loses at this point. You've made it clear that nothing I can do at this point will result in me being able to extract more money from you in the future.

My point is that you shouldn't say that you'll never do business with them again (even if it's true), only threaten that you'll never do business with them again, or better yet that you'll never do business with them again and you'll

As someone who wields a 7-figure IT budget and has dealt with Dell I agree. No surprise at all. I'm actually surprised that they still have corporate customers. If those of us with large budgets making high volume purchases have a hard time dealing with them then what kind of service does a small business get? I tried Dell out due to their slightly lower prices but I've learned my lesson and I'll stick to Lenovo.

Lenovo sucks as bad if not worse. I spent 6 figures on their laptops and every single T61 ended up having to have the panel replaced every 6-18 months. In every case the backlights failed, we have had them all replaced as they end warranty coverage and will discard them when they fail again.

I will never again buy a Thinkpad, sad to say but dell has done better than that. I used to only buy thinkpads.

I don't wield any kind of budget at all, but my company has a few dozen divisions internationally and uses Lenovo exclusively for the few hundred notebook users we have... as an end user I'm pretty damn happy with my W500. I've had a ThinkPad series device for the last 11 years and have come to like them enough that I bought one for my personal machine at home.

Gateway had the same vendor. I was working in one of their shops and anything with an AMD processor was likely to show up a couple times.

That's one advantage Dell had over GW. GW had built up a reputation that if there's a problem come in and we'll fix it under warranty, then these motherboards hit and there was no way to keep up with volume.

So they fixed it no problem if they could swap the processor out and AMD took the financial hit, but if it was their own low-bidder motherboards bought from a fly-by-night manufacturing firm in Hon Hai that doesn't exist anymore, you were getting stonewalled.

It's not a stereotype. It's really hard to find corporations to do business with that aren't engaging in that sort of behavior.

Part of it probably comes from the fact that our regulators don't regulate, meaning that the only way that a lot of that stuff gets punished is through class action suits. But, until there's a credible suit you can't just subpoena records from random companies. And so there's a greatly reduced risk in the short term of anybody noticing. Hence this sort of bad behavior, since most

It not just "bad" caps. My 42" Samsung TV died, not because of defective caps in the power supply, but because the caps were inappropriately rated. They were 10V-rated caps in a 15V circuit. It was just only a matter of time before they died.

And thankfully I found articles on the cap issues before I plunked down $999 for a new TV. $2 in caps, and 45 minutes of my time solved the issue.

Well, $2 in caps, $23 in shit I didn't really need to cover minimum orders, and $20 in "overnight" shipping all because jASSper, TX was a shithole of a town and the local Ratshack doesn't carry anything but TV antennas and Verizon phones.

Dell was not the worst by any means. IIRC, the worst as far as caps leaks from those days was probably MSI. Asus was mostly OK (only some MB models affected) and I have not seen a single Via EPIA motherboard exhibit any of these problems.

I am still using about 7-8 Vias from those days quietly shuffling files as a small file server or serving as a media center. Not a single one of them is showing any problem and some of them have been through thermal hell in cases with failed fans.

Quite true, from my time working in staples, I can say it's pretty clear. Cheapest > best 95% of the time. No matter what I would say or do, I couldn't talk people out of buying the $300 machines during the horrific era when manufacturers were releasing vista on systems with 512MB of ram.

"Yes the e-machine has windows vista, and is $300, but I have to warn you, the hardware on it is far to weak to run what they have installed on it. If you buy this machine, you can expect it to crash several times a day

You only got yourself to blame. For every story of a customer ignoring the good advice of a sale person, especially with regards to tech purchases, there are 10 stories of a sale person reaming a customer with some bullshit about the need for $100 gold plated, oxygen free audio cables.

Now I'm sure you're one of the good ones, but your line about crashing is pure FUD. Run painfully slowly, yes. Crash several times a day, bullshit.

Sadly what he said about crashing ISN'T FUD, because I used to run the "go to" shop that dealt with all the Worst Buy disasters. You have to remember that NOT ONLY were you dealing with Vista, which frankly anybody selling it on less than 2GB needed their head examined, but you had all the OEM bloatware ON TOP of Vista. I've seen them run 20 minutes and crash hard simply because some OEM bloatware fought Vista for every KB of RAM until it went tits up.

You're being facetious, but I think you are actually completely correct. The free market only works when people are informed. If everybody had perfect knowledge of the ins and outs of the things they were buying, then perhaps we could have a free market utopia where the invisible hand of the market automatically corrected everything.

The problem is that we live in the real world and most business deals involve some kind of information asymmetry where one party (usually the seller) knows more about the produc

Yes; what we need are government regulators that Dell can pay off so that these stories never actually make it out* and give buyers the opportunity to react with their wallets.

Sarcasm aside: government regulation is good and helpful to a certain extent, but it doesn't solve all problems. Why? Because the government is made up of the same people that make up companies, and they can be bought, corrupted, and act unethically. And, unfortunately, we can't easily "boycott" the government, whereas we CAN refuse to buy Dell products, if we so desire.

Unfortunately, I'm only a doomsayer, I don't have many good ideas in this way. I would like to say that I don't know many people who actually think there should be a 100% free market, just like I don't know many people who think we should have a 100% regulated market (i.e., no freedom in the market at all). Arguing against an exaggerated position of people who suppor ta "Free market" may not help much. Or, I may just be ignorant of the folks who actually advocate a 100% no-regulation-whatsoever market...

* Because then a lawsuit against Dell would also involve the regulators and regulating governmental agency, which basically would mean that another group of people would be at risk and raise the incentive to hide the documents/defeat the lawsuit. Examples, perhaps? The recent coal mining stuff and BP. Those had government agencies attempting to regulate them and whatnot. Fancy that, the ones that were overseeing BP were corrupted. Shocking.

I don't know where you get your misguided opinion, but nearly all US government employees are hard working honest people who won't be bribed or corrupted.

There isn't a perfect solution, but government regulators are generally good at their job. Sadly, it's a job that, when your doing it well no one notices. And then when people wan't 'taxes cut' it's the people in the background doing work that gets cut.

The Phrase 'cutting taxes' has done a lot to hurt this country from functioning.

When people started talking about bad caps.. certain MOBO makers went out of their way to label their boards as "Premium Capacitors only", or caps from japan, or anything to show they weren't infected with the bad ones.

You act as if it isn't the status quo for most corporations to behave like that.

The reason the GP stated that is that if we had real regulation of corporations like that it wouldn't have gotten to that point. But, because the free market results in supposedly better products, service and prices we don't need to subject them to oversight.

It's not a straw man. "Free market" has one meaning: a market with zero regulations. If libertarians don't want that, then they should stop saying "free market", and I really wish they would do that because it promotes ideology over moderation. If we drop the ideology that "regulations are bad", then we can get to the real adult work of deciding which regulations are net good and which ones are net bad. Do we have too many regulations today? or the wrong kind? Perhaps, but it's hard to discuss it because the ideologues are always ruining the conversation by shouting about free markets.

But, libertarianism is an ideology. It can never be anything else. Libertarians will never change, but the moderate alternative is simply "conservative". I don't consider myself conservative in most regards, but at least they are realistic instead of ideological.

Markets are good; free markets are bad. Moderation is good; ideology is bad.

IMO, the important thing about this article is they finally reveal the source document their claims came from. This is important, especially because of the kind of comments the last Ars Technica article about this lawsuit [arstechnica.com] had.

This isn't just capacitors. I almost stopped doing business with dell completely after a client came to me with a clearly failed nvidia chip on a model that had the warranty extended for just that problem. They had called dell during the warranty period and were told it was an issue with the OS and they needed to reinstall. They trusted dell. They reinstalled. They updated their firmware. The computer lasted another few months with the problem getting progressively worse until there was no video at all. I tested the system and determined definitively that it was the nvidia chip and asked dell to replace the board. I was given the runaround being told how do I know and its out of warranty. I pointed out that the warranty had been extended and my customer had called them during that timeline and was given bad information by their support team. They fought it and fought it and fought it some more until I called the rep that I do large orders with for corporate clients, and apologized to him that I would not be ordering anymore servers etc. from him. I explained the situation and was called back by dell corporate the next day offering to swap the bad board for a refurbished one. It solved the problem, but it really shouldn't have to go that far. I love using dell servers, but having experiences like that do not make me want to use their products.

I'm still seeing these burnt up Nvidia chips get replaced with the same identical board on warranty repairs, and still finding busted caps on certain (cough gx270 gx280 cough) models.

We had a very large number of GX270's. Within 4 years, 2/3rds of the power supplies and 1/2 of the motherboards died from bad caps. They didn't even flinch when I called and said the MB had caps leaking brown crap. They were happy to ship a new MB and let me swap it even though we had the on-site service contract. The problem was that Dell was shipping replacement parts that would have the same problem within 2 months. I stopped calling on the power supplies and just bought new non-dell branded ones.

They had to know they had a widespread problem and that the parts they were shipping would have the same issues. They were simply trying to use up their spare parts inventory and string the customers along until they were out of warranty service contract. That's dishonest.

They were simply trying to use up their spare parts inventory and string the customers along until they were out of warranty service contract. That's dishonest.

Actually, I think from a legal perspective, Dell was colouring between the lines, if the contract was for Dell to replace the system as often as it failed within the warranty period.

You might complain that it's not ethical given the business image that Dell put forward. Obviously they decided that their short term reputation with share holders was worth more than their long term reputation with customers.

Given the size of the problem (across the industry) I'm not sure there were enough good capacitors avai

I have had some mixed experiences with Dell. Some great. Some not-so-great. But you know? They are still better than the rest... even if they are only a bit better. I have grown accustomed to dealing with their support people -- I know how to answer questions to expedite the results I seek. I am also certain to buy NBD support for machines I care about -- servers and laptops mostly. (Accidental damage for laptops every time!)

I have had to deal with the capacitor issue in the past. Maybe my company w

You've set your price pretty low there. All it costs to make you go quiet and keep buying $000s worth of their kit it one motherboard that they should have replaced without being threatened anyway. You really taught them a lesson there and I'm sure they'll keep learning from it with every future order you make from dell corporate.

Nothing personal, but I've never understood this sort of reaction. If I get to the point of threatening a company then it is already too late. I don't threaten to take my business

The US Federal Government buys more Dell machines than any other major customer. And Dell sucks, really really hard.

Sure, their server hardware is OK, but it's just off the shelf stuff which is more expensive than a lot of competition, including the superb Supermicro. So, the only conclusion is that Dell has employees that suck a really good dick.

Now this comes out. I wonder what the total damage done to the taxpayer was? Probably in the hundreds of millions when you figure in the lack of services caused by downtime, contractor handoffs of parts before they actually get the problem fixed, and subsequent testing which is mandated at many facilities.

Political parties do much the same thing. The so-called base voters who would never consider voting for the other party (or staying home) can be and generally are ignored by candidates because they know their votes are secure.

Loyalty is a terrible position for a customer (or voter) to take. If you want results, insist on getting them up front, before you fork over the cash (or votes, or, in our political system, both).

Well telling them the truth was a never an option, that would had involved telling them that "Sure we used cheap or substandard components in our machines and you pay a premium for a quality product... Sucks to be you!"

That was option 1 option 2 would have been to pay a bit more for better quality components in the first place and while I would say pass those costs on to the customers they already had done that.

I haven't been able to recommend Dell to home buyers for years now to be honest. If your a corpor

Dell may have been more customer-antagonistic than other manufacturers, but even alleged luminaries in the business were tainted by this issue.

My first Apple base station was based on a Lucent design that Apple put a graphite-colored plastic enclosure around. Naturally, the Job/Ivs-ian approach to mechanical design did not allow these base stations to have ventilation holes in them, even though they had a comparatively big internal linear power supply and were using a 486 chip. Combine that with all the remaining hardware and you had a nice hot little box, especially if you used the dial-up modem. A year later, and the marginal Lelon capacitors powering the the base station started bulging like Champagne corks or popping off altogether.

Naturally, Apple told its customers that the they were SOL if the unit was out of warranty after a year of ownership. Those who had AppleCare warranty could get refurbished units - usually in marginal cosmetic condition - and only if they mentioned that AppleCare covered attached peripherals. Apple never proactively contacted owners of graphite base stations to acknowledge the issue and to point owners towards repair options.

I got mad enough to investigate the issue, discovered the bad capacitors and created a web-page to teach others how to replace them or have service providers replace the capacitors for them. Not that hard to do. I also gave folk instructions on how to add ventilation holes to help these poor base stations cool better. The Lucent design covered much of the board with an EMI shield, which exacerbated the thermal problems - it's like encasing the electronics inside two heat shields.

As the issue affected more and more customers, Apple started a non-publicized warranty program that allowed customers outside the warranty period to get their unit replaced - but only if they knew what knowledge-base article to point the Apple drones to. Naturally, just as the program appeared one day, it also disappeared after a while - without a press release, notice to customers, etc.

All along, the typical answer from an Apple phone-drone was that they had never heard of the issue before. So, if you did a little digging at Apple, I would not be surprised if the SOP manuals for phone-drones include the 'suggestion' that every issue reported by an irate customer is 'unusual', 'never heard of before', etc. It's one way to mollify customers, especially those who don't know of the myriad of other customers affected by the same issue.

The only times I had Apple admit something outright was with the Santa Rosa graphics chipset problem, and probably only because by MacBookPro was covered under AppleCare. However, by then, a lot of of other folk had already been affected by this issue and NVIDIA was presumably paying for the PCB repairs. So I'm not sure if I can give Apple a pass on that one either. The first sets of customers were probably told that unless the unit was under warranty or AppleCare that they'd be buying a new motherboard and paying Apple for the privilege of getting it installed too.

Would the base stations have lasted longer if Apple had elected to use name-brand capacitors instead of Lelons? Perhaps, but any electronic appliance last longer with lower operating temperatures. Sadly, this is an issue that seems to continue to haunt Apple - a desire to design pretty enclosures whose thermal performance is at the borderlines of what the electronic hardware can tolerate.

We had almost 90% of our GX270 desktops (about 200 deployed at the time) fail due to the bad cap issue, and Dell Canada repaired every single one within 24 hours, no questions asked. Since then we’ve had near zero issues with any Dell laptops or desktops (over 1,000 deployed and in use), and the few failures we’ve had since then Dell has fixed without any hassle.

Dell Canada never gave us a hard time over repairs for the bad cap problems. Whenever we placed a support call all it took was a ment

I have to post a big Me too on this one. Never an argument, or a crazy request to reinstall software or try pointless remedies.

Nowhere near as many systems as you but exactly the same response. They fixed units that failed, took me about 1-2 minutes on the phone to go over the details and the rest of the time was just waiting for them to fill out the dispatch forms.

I used to work as a systems administrator for the company suing dell, AIT, and to be honest from the setup we were required to use you could not tell whether the problem was because of dell or the high heat on the machines. The problem stems from the owner of the company's desire to cut costs anywhere he could. He stopped buying actual servers and went to buy desktops and sold dedicated hosting services on them as though they were servers. We would have groups of 3 pizza racks stacked with 12 of the VCR dells per shelve. The heat from all the machines was terrible, we even had plastic melting on some of the machines.
To cool the system we had 2 used industrial ACs, that were always breaking, in addition we had 3 of the large stand up fans that really did no cooling at all.
What is really ironic about this suit is that the company at the time advertised nightly backups on all accounts, however only about 1% of the customers, the ones on our netapp, was actually backed up. For the rest the company would not buy the hardware to actually backup customers.

But the real problem with Dell is their entire business model. They manufacture nothing. They buy components (motherboards, etc) from the lowest bidding Chinese OEM, who manufactured those components using parts from the lowest bidding Chinese OEMs. The bad capacitors aren't Dells, fault. They aren't even the motherboard manufacturers fault. The bad capacitors are the fault of the company that made them. And unfortunately, you can't tell they are bad until they fuc

...because this post is in response to those who espouse "the man fucked us over again" attitude you see a lot of here.

Just for a minute, step outside your cubicle and put yourself in charge of Dell.

First of all, you have no choice but to manufacture your laptops in China, Viet Nam, Malaysia or some other God-forsaken country because it takes a lot of labor to assemble your products, all your competitors do the same thing and people want a full featured laptop for $599. If you stamp a "Made in USA" sticker on the box and charge $899, a price at which you'd lose money if you built it here, your product sits on the store shelf like an old turd.

Secondly, you have to source subcomponents like capacitors (or in the case of Mattel, paint for Barbies) from a network of vendors within the country you're building in. This sub-supplier network could go 3 or 4 or more companies deep by the time you get down to the raw materials used to make the parts.

Of course you got samples, tested the shit out of them, and insisted that all the components and sub components come only from approved suppliers. You put in incoming QC tests to make sure the parts adhere to spec, but it simply isn't possible to test every single component every single time they come in. You've got ongoing WIP tests at every step of the way to make sure the subassemblies and end product stays in spec.

Then some peasant in Northern China decides to send some bad raw materials used to make the capacitors used in the Dell (and other) machines up the stream to the capacitor factory. The factory making the caps has no idea the material is bad and the caps test out fine. They only fail over time so they pass any and all incoming QC tests your factory has put in place, and the end computer they go in tests out fine before shipment.

Then you get the first return from the field. And then the second. After root cause analysis, you finally recognize that the capacitor is faulty, and basically you then say Holy Fucking Shit. Because the lead time to get the caps was 12 weeks, and by that time you've got 150,000 suspect machines in the field, another 150,000 in work in process, and another 150,000 worth of parts in the parts bins waiting to be assembled.

You now have to figure out how and why machines fail, over what time span, and under what usage patterns. Will this thing burn down a customer's house!? How badly are you fucked in terms of warranty - how many machines will fail before the warranty is out, and what is the company's exposure? What about all the stuff in the supply chain - do you pay to have it reworked or do you deem the risk low enough that you continue to build and ship?

Then, you start the customer triage process. Who are the most strategic customers that will have the most downside due to the failure? Do we do a proactive recall, or wait until machines fail and come back in? if we don't do a recall, do we alert the public or just wait and deal with the failures as they arrive and hope it doesn't get any worse?

Of course, as part of this process, you've found the mother fucker in China who screwed you, had him shot, then put in tests to make sure this never happens again, and tried to get some sort of money out of the sub supplier to cover the millions you have at risk due to this problem. Good luck with that, by the way.

Then some douche bag on Slashdot calls you a greedy pig for not replacing every machine without question.

Welcome to the world of the manufacturer.

And by the way, the above if GROSSLY simplified to enhance comprehension.

One thing to remember in this case, however, is the machines in question were not the Dell Crap line machines, they were the premium-quality Optiplex line, where you pay more to get a reliable machine for Enterprise users.

And the bad caps? Not the work of poor QC from a "Chinese peasant", but industrial espionage from some Taiwanese capacitor firms who had engineers steal a formula from from a company in Japan, but got it wrong.

And Dell bought low-end capacitors from cut-rate suppliers. They are not the first to make this mistake, but on your premium line, where you charge premium prices, you should be buying name-brand components. Good electrolytics are expensive.

Sometimes threatening is not enough. Some years ago, I realized that my company was paying way too much for our web and email hosting, so I called the provider and threatened to move unless they reduced the price. The provider would not budge.

So, I spent some time configuring a replacement and called to cancel the service. Now, the provider wants to offer a discount! I told them where to shove it, because I had already invested the effort in replacing them.

Don't you alleged iPad jockeys have better news to post than this OLD news?

This is NEW news (or at least new information) relevant to the old news. I have no idea what connection you are trying to make to Apple here.

Perhaps, you might cover the intrusiveness of TSA "security" [youtube.com].

Because that is never covered here, and the following articles from the last couple of weeks are figments of my deranged imagination:
http://yro.slashdot.org/story/10/11/16/1723233/US-Marshals-Saved-35000-Full-Body-Scans
http://yro.slashdot.org/story/10/11/13/2229222/National-Opt-Out-Day-Against-Virtual-Strip-Searches
http://news.slashdot.org

I'm not exactly sure why I was modded troll for this. The problems that HP had with DV2xxx and DV6xxx notebooks, particular with the nVidia chipsets is well known, as is HP's refusal to properly deal with widespread issues of overheating and damage.