Hat's UPS Explodes!

I've been a "roll your own" UPS guy for about eight years now, but today (well, yesterday), it exploded!

I mean really exploded. There was fire and lightning and stuff that the deep south would blame on inclement gods.

How had Hat so badly offended the gods?

It all started when I replaced the battery bank back in June. I use an array of eight 24V truck batteries, each 130 amp-hour rated, so a total of 3,120 amp-hours, or 74,880 watt-hours. For its rated 500 watt load, this gives 149.7 hours runtime, which jives well with my four-day goal. It's six and a bit days. Before that, I had just two batteries for much less runtime.

That was my first mistake. You see, charging the fuckers was done by a circuit meant to charge just two 24V batteries. This was okay when the batteries were full, as they didn't need much juice. The power transistors (2N3055 things) were rated for 40 amps so I didn't see the need to protect them (the rating is hopelessly optimistic). You see where this is going, don't you?

There was a power outage, a fairly major one at 9 hours overnight. The UPS tripped its relay and engaged. Systems continued to run (this is not an inline UPS) and everyone's happy. 9 hours later, mains came back on and the UPS disengaged and began charging. 9 hours and 2 minutes later, the UPS' electrics box exploded.

With the batteries being a bit discharged, they wanted juice and would accept a whole load of it. I estimate that the charger peaked at around 40 amps before exploding. The four 2N3055s resulted in just one remaining intact two being discoloured on the back, and one plain shattered outwards on the reverse. I've never before seen this failure mode in a TO-220 transistor. The circuit design meant that a failed transistor would fall over onto the remaining transistors.

As it was hopelessly overdriving them, the first failure meant the remaining three were being hit even harder and base current went through the roof, as it was trying to compensate for something that didn't exist anymore. As the transistors failed, the last transistor was being hit by an enormous base current, so it exploded.

Well, at least the whole shebang was minor enough to have you posting again here a day later

As someone who's about to roll his own in the short term future, I'm glad that this is not among my oversights in the planning.If you use truck batteries, I suspect those are wet cells. Be glad your charging circuit blew up so soon. If it would have failed in a different way and provided a too high charging voltage, you could have had a box full of burning hot stuff and batteries emitting H2 and O2. THAT would have been an explosion! With gallons of sulfuric acid!

I guess you now are painfully aware of that, but you always want to fuse such circuits one way or the other. A not too technically sophisticated and reliable "fix" is to just put a lightbulb of appropriate resistance in series. 12V halogen bulbs are low resistance when cold, can dissipate a lot of heat and are cheap to get. For that many kWh of wet cells I'd put in some more effort, though.

Not because they don't perform a useful purpose, but because they end up being a crutch which you think is there, and when push comes to shove, it's not really.

If you want a proper UPS (like yours), you need a hell of a lot of ampere-hours stored -somewhere-. A proper UPS weighs a lot, and has very restricted run-times; especially if you're hooking up more than one device, and including a computer monitor.

My desktop has a 500watt power supply, and I reckon when I am playing Planetside 2, it's using 400W+ continuously.

I have a 32" HDTV as my main monitor, and a 23" Dell IPS monitor as a second; I run USB mouse, USB keyboard-extender (Logitech G13), USB keyboard, USB screen-calibration wand, microphone, headphones, FitBit dongle/charger, external SATA drive connector, as well as the router and the ADSL modem; fortunately my ZFS box with its multiple HD's is off 99.99% of the time, so it's not an issue.

Running that lot -reliably- on UPS would mean a beastly expensive UPS solution. Since I'm not living in a third world economy (yet), electricity supply is almost perfect (regular grid maintenance being the most common reason for electricity shutdown). Lightning strikes and other power-failure causes account for way less than a percent or so of "24/7" uptime, many months go by with 100% uptime (I think the record is 10 months).

I don't like the idea of spending big on UPS when it's only usable for a short period of time, and it's just another -source of problems- in the ongoing scheme of things.

Having said that, if I DID have a functional, gutsy, flawless UPS, I'd be a tiny bit happier in my life. So reading about UPS designs which are functional, practical, and not very expensive, is definitely attractive.

(four new 24V truck batteries would cost between $US600 and $US900 here in Australia)

and I reckon when I am playing Planetside 2, it's using 400W+ continuously.

Out of curiousity, what kind of box do you have? 400W these days is usually a fast Ivy Bridge and a GTX 680 or something. If you really are into that territory, yeah, it's not cheap.

That said for me I just want 5 minutes' worth of power to do a clean shutdown, and for that, www.refurbups.com will sell me a refurb Liebert or APC Smart-UPS tower that'll do that for what, $300 or $400? (no idea on what that costs in Australia)

Again depends how much runtime you need... if you need serious runtime then Hat's solution is a very good one (if you are handy with electronics and don't mind the fire risk ), but if you only need a few minutes of runtime, then you can buy that off-the-shelf...

There are no pictures of the carnage and destruction. This thread disappoints.

So basically, you'd been pissing on fate's leg, for over 6 months, and fate looked up, was displeased, and tazed you.

What's the plan to regulate the charging so it doesn't release the magic smoke (and that burnt smell) again?

Also, why the batteries and not a multi-fuel generator?

I'm a bit curious; as you might expect after the issues around here a few months ago, lots of discussion @ work about generators, UPS' and the like.(Also waterproofing, mold, types of drywall and whether or not people living in a consistently flood prone area that keep rebuilding in the same spot are resilient, stubborn, or insane.)

I don't want to go into details before being sure that this stuff works as planned. I'm still waiting for an answer from a manufacturer of DC-ATX PSUs. But basically I ripped off Google's practice of just stuffing a battery with the PSU in a box that runs off 12V only. Except I want to be doing it with ~18V, which is three 6V lead cells in series.

It's really a by product of wanting higher efficiency PSUs for my rigs that have a very low idle power consumption. I ended up with the idea of using laptop PSUs and DC-ATX PSUs from there. And with a nice low voltage DC stage, adding buffer batteries is not a big deal.

What I've planned could be done for 400W, but I don't know about DC-ATX PSUs that can do 400W or more sustained, let alone with non stabilized input voltages. I also don't plan to power any screens. For those a normal UPS should be way easier, as their power supplies are anything but standard and interchangeable or with specs for their output available.The boxes this will be for could be administered via ssh or AMT from a laptop should the need arise. Network gear (switches, routers, modems) will also be supplied by the UPS stuff to prevent interrupting iSCSI or NFS connections.

The scenario this is intended to protect against is what I had last year, when some dimwit of an electrician fucked up a closure. It was three weeks of intermittent voltage drops, brown outs and outages in the second to minute range - with a 5 hour outage at the end, when smoke came out of the hole the closure was in.Electricity supply here is usually really good and anything above single minutes of outages is extremely rare, so I don't plan for that, but it's just a question of how big the batteries you buy are.

The question is what your UPS needs to protect from. Flaky Voltage and brownouts, overvoltage and spikes, short power loss, long outage? How long?How technically skilled are you? What can it cost? etc etc

For a complete server room or just one well populated rack that has to stay powered I agree that batteries should just bridge some minutes to get a generator up with another 5-10 minutes for a controlled shutdown if that fails - or to try the backup generator if it's that crucial to stay up.

If we talk about 500W, though, some hours on battery are doable and might be more viable than a generator. The battery in my car should do just under 2 hours at 500W if I guesstimate the capacity degradation right. Two of those (preferably in a gel version) should do fine for 4 hours. Make it 3 hours if they get a little older.Now tell me how you get a diesel backup in the third floor of an apartment building or the second floor of your house.

If we talk about 150,000W, things are different again. One company I worked at had their space rented in a big office building that had a giant rotary UPS in the basement. That thing was a beast.

The power transistors (2N3055 things) were rated for 40 amps so I didn't see the need to protect them (the rating is hopelessly optimistic)...I estimate that the charger peaked at around 40 amps before exploding.

Pardon my inebriated brain from saying so, but doesn't this suggest a failure of design rather than hardware, as they gave out precisely when and where they were rated to do so, and the rating is therefore not, "hopelessly optimistic", but rather precisely "what it says on the tin"?

The power transistors (2N3055 things) were rated for 40 amps so I didn't see the need to protect them (the rating is hopelessly optimistic)...I estimate that the charger peaked at around 40 amps before exploding.

Pardon my inebriated brain from saying so, but doesn't this suggest a failure of design rather than hardware, as they gave out precisely when and where they were rated to do so, and the rating is therefore not, "hopelessly optimistic", but rather precisely "what it says on the tin"?

I've a 6KVA diesel generator, it'll run on a decent load for nearly 24 hours on a 18 litre tank of red diesel (80p a litre) I can run computers, central heating, lights, and either one ring or the oven on the cooker.

The power transistors (2N3055 things) were rated for 40 amps so I didn't see the need to protect them (the rating is hopelessly optimistic)...I estimate that the charger peaked at around 40 amps before exploding.

Pardon my inebriated brain from saying so, but doesn't this suggest a failure of design rather than hardware, as they gave out precisely when and where they were rated to do so, and the rating is therefore not, "hopelessly optimistic", but rather precisely "what it says on the tin"?

There were four of them. Badly heatsunk, it turns out.

In your OP, you mentioned that they were TO-220 packages. Did you mean TO-3? Seems like that is the right choice for this application. 10 amps through a TO-220 is way too high... A TO-3 will take 15 amps collector current, I think.

The batteries are fine and will remain so. No generator because it's rather hard to run one indoors, while I can stick the batteries in the loft on a few crawl boards above the fibreglass insulation and run little risk of killing myself.

The idea is to rejig the charging circuit. Its front end decides if current is going in or coming out, that bit is good. The back end is what routes power around. It's now duty-cycle limited to 20 amps using an oscillator which trips in at about 15 A and will fail-safe at 30A using a polymer fuse. Each battery now has its own 2N3055 and its own route to juice, the charge controller turns the power transistors down (or off) as needed. An overcurrent on one output will have the effect of throttling output across the entire array to ensure safety. Finally a number of 40 amp non-resettable disposable fuses will protect all outputs.

This has the side effect of removing the ability to detect battery voltage from the circuit, so an on-battery device does that, shutting off incoming power when 26.4V@5A is reached. This samples battery voltage by interrupting the charge every 5 minutes and dropping a load of 5 amps across the battery.

Any suggestions here? I'm quite inexperienced with designing lead-acid charge controllers, any way of getting the "I'm charged" state without hitting five amps every so often?

With luck, this thing will handle my future idiocy.

(Also, does anyone know how to step-up DC voltage? Specifically, 15V to about 25V? More efficiently than the dumb inverter>SMPS route?)

Have you tested a sample of your new transistors to failure? I have read about 2n3055 being substituted for other things (http://sound.westhost.com/counterfeit.htm) but the margin per part would not have to be high to make it worthwhile.

I can't really help you here, but as you're playing around with lead batteries you might have stumbled upon a bit of information I'm currently trying to find.

Do you have any data for the relationship between voltage and final charge level for constant-voltage charging lead acid or lead gel battieres?Or asked differently: if I take a rather empty 12V battery and put it on a 13.2V source, how much of the capacity it has when charged with the usual 13.8V do I get, after an indefinite amount of time?I've got all kinds of datasheets and crap here, but seemingly nothing answers the question what the minimal charging voltage for lead cells is to reach their rated capacity and how much you lose when the voltage is gradually lowered...

I've got all kinds of datasheets and crap here, but seemingly nothing answers the question what the minimal charging voltage for lead cells is to reach their rated capacity and how much you lose when the voltage is gradually lowered...

Charge to 13.2 volts. That IS your minimal charging voltage, and about the safe voltage you can hold them at to trickle.

If you only charge to 12.8 V, you get about 60% rated capacity.

You can't measure a battery's capacity using only voltage. You need to sample the current too. As a battery fills up, it begins pulling less current, but its terminal voltage doesn't rise. Just use a voltmeter, an ammeter and sample every 5 minutes or so, then use simple area under graph methods to work out capacity.

Quote:

Have you tested a sample of your new transistors to failure?

There are a LOT of alternatives to the 3055, but that said, I'm going to ratchet up a test for six of them tomorrow (I bought 120...) where I'll take them to their 15 A rating and beyond, up to 20 A, on active cooling. I should never exceed 2A per transistor, but you never know.

Charge to 13.2 volts. That IS your minimal charging voltage, and about the safe voltage you can hold them at to trickle.

If you only charge to 12.8 V, you get about 60% rated capacity.

You can't measure a battery's capacity using only voltage. You need to sample the current too. As a battery fills up, it begins pulling less current, but its terminal voltage doesn't rise. Just use a voltmeter, an ammeter and sample every 5 minutes or so, then use simple area under graph methods to work out capacity.

The specific batteries I wanna use are 6V (will use 3 in series) and their charging voltage for trickle use is 6.8-6.9, which would be 13.6-13.8V for a 6 cell version.Obviously charging with only ~1.95V per cell is going to get me about 0% capacity. But between that and 2.3-2.4V per cell, how does the final capacity of a cell change with varying voltage? Is it linear? With 13.2=>100%, 12.8=>60% and 12.2=>0% this seems sensible. But I have not found any source confirming this.

The background is that I do want to get rid of a charge controller in my setup entirely and just current limit the charging process.

I also don't care about estimating the batteries' current capacity and am aware that this is practically impossible to do solely based on voltage. I'll just have the UPS send a shutdown signal at around 16.2V terminal voltage and have it shutdown hard some minutes later. According to the batteries' datasheet a terminal voltage of 5.4V is still in the operating range at a way lower current draw then I'll ever have and even at the highest current draw it should not drop that low before about 75% of the capacity has been depleted.

I could design a controller, make myself a fuel gauge, add occasional cycles of depletion and strong charge to make the batteries last longer and all that. The goal is to reduce the complexity of the setup to a resistor and a diode, though, without losing more than maybe 50% of the potential runtime and without killing batteries too quickly.

I have no idea. But the manufacturer of the picoPSUs also has models for automotive use cases that handle a wide range of input voltages. The M4-ATX I'm already using takes anything from 6V to 30V, with degraded performance towards both ends, can actuate a power switch and has a freely definable input voltage for shutting off a system this way. I also have some crazily good 20V laptop bricks around. So I want to just shove in some batteries that will be trickle charged and take over in case of a power outage. Shut down via signal to a power switch also works well enough for me.With my main home server, the network stuff (switch, router, modem) and the NAS buffered this way I'd be set. Most state critical stuff happens inside VMs and a laptop is always around for the cases were it's not just a minute of outage. The filtering aspect should also be covered to a large degree, with the M4-ATX being designed to work with the noise of an automotive DC system and a lead acid battery in parallel to dampen any spikes or outages making it through the 20V brick.

Of course this is not a generic solution and can't replace a normal UPS for a workstation that would also cover the screens, but it perfectly fits my use case. Especially because the additional power draw would be almost negligible (trickle losses of the batteries) and the combination of those 20V PSUs and the M4-ATX is surprisingly efficient. By guesstimation I'm close to 90%.

But the manufacturer of the picoPSUs also has models for automotive use cases that handle a wide range of input voltages. The M4-ATX I'm already using takes anything from 6V to 30V, with degraded performance towards both ends, can actuate a power switch and has a freely definable input voltage for shutting off a system this way. I also have some crazily good 20V laptop bricks around.