EEVblog Electronics Community Forum

Electronics => Repair => Topic started by: Lorenzo_1 on July 07, 2020, 03:37:40 am

Title: APC SmartUPS 1000 failure (FIXED)
Post by: Lorenzo_1 on July 07, 2020, 03:37:40 am
Have a 3-4 year old SmartUPS (bought as new/old stock). 240V circuit breaker tripped and when trying to restart UPS it was cycling and shutting down repeatedly. Then it blew out MOV on a connected powerboard. Stripped it down - pretty heavy dust behind one of the two fans that cool MOSFETs.  Would have reduced airflow.  Quick inspection showed blown-out 400V 10uF cap (pics attached). Fortunately plastic sleeve contained the contents and kept board clean.  Replaced and everything started up fine - self-test OK.  Thought I'd pass this on as data on these units is hard to find and others might encounter similar problems.

Two comments:  Vacuuming dust off the outer covers regularly may not be sufficient to prevent overheat - I'd suggest pulling top cover occasionally and cleaning thoroughly (noting dangerous voltages inside). Also getting top cover off can be a challenge - it slides into place with short slotted grooves either side and clips into place at the rear of the unit with bent tabs.  Had to get a plastic lever between body and rear tabs to press it forward an inch or so to get it to release.  Quite a struggle after some years in place and being unsure what was holding it.
Title: Re: APC SmartUPS 1000 failure (FIXED)
Post by: james_s on July 07, 2020, 04:43:16 am
Wow, I don't think I've ever seen one of those film capacitors blow up like that. I wonder if it shorted internally? You said it blew the MOVs in a power strip? That implies that the output voltage went way high, it's likely that capacitor was part of the filter on the output so the high voltage may have blown the capacitor too, that could well be a symptom rather than the cause.
Title: Re: APC SmartUPS 1000 failure (FIXED)
Post by: Lorenzo_1 on July 07, 2020, 02:58:53 pm
I am interested in what might have caused failure, but it’s beyond my skillset. I think the cap must have failed first and before the over voltage b/c after tripping breaker UPS would go through startup repeatedly but fail. It popped the MOV quietly (only one of three blew) - no discernible noise out of UPS and just a small crack/flash out of power board. Seems it might not have been too violent in UPS as sleeve that contained the debris was not especially robust. Looked more like ooze than explosion. Might airflow obstruction cause such a failure?
Title: Re: APC SmartUPS 1000 failure (FIXED)
Post by: TheMG on July 07, 2020, 03:12:39 pm
Very different circuit board from what I've seen in quite a number of APC Smart-UPS I've opened up. What's the exact model number?

Anyways, depending on the inverter topology, with the failed filter capacitor it could have subjected the load to high frequency switching ripple and transients which is probably what took out the MOVs in the attached power strip. It's also possible the unfiltered inverter output caused the feedback loop in the UPS to go bananas and output excessive voltage.

Anyways, I doubt overheating of the inverter MOSFETs caused this. That would most likely have resulted in MOSFETs going short-circuit. Film caps normally run pretty cool as they have very low ESR so lack of cooling of the capacitor is likely not it either.

Although quite rare due to the self-healing properties of most metallized film capacitors, catastrophic failures sometimes do happen.
Title: Re: APC SmartUPS 1000 failure (FIXED)
Post by: Lorenzo_1 on July 09, 2020, 02:04:11 pm
Thanks for the feedback re failure mode.  UPS appears to be SURT1000XLI based on barcodes etc on rear. Good news is it's been completely stable after that simple repair. I think that's the first time I've had something fixable with only a cap replacement.
Title: Re: APC SmartUPS 1000 failure (FIXED)
Post by: james_s on July 09, 2020, 07:08:35 pm
I've fixed many dozens of things with only a cap replacement, although this is the first time I've seen a film cap like that blow up.
Title: Re: APC SmartUPS 1000 failure (FIXED)
Post by: TheMG on July 09, 2020, 07:11:00 pm
Ah, a double-conversion UPS. All the Smart-UPS series I've ever come across were line-interactive, with a bulky low-frequency step-up (and buck/boost) autotransformer.

So yes, it makes sense that this capacitor serves a critical purpose in the functioning of the high frequency PWM inverter, both during online and battery operation.
Title: Re: APC SmartUPS 1000 failure (FIXED)
Post by: elecdonia on July 22, 2020, 07:35:02 pm
It's likely this exploded capacitor is connected directly across the 120V 60Hz (or 240V 50Hz) AC output signal coming from the DC-AC inverter section.

This capacitor filters out the harmonics and pulses/spikes generated by the high-frequency switching type inverter. These inverters use a full-bridge switch array (four MOSFETs) with PWM (pulse-width modulation) to simulate a sine wave AC output signal. The raw full-bridge switch output has large amounts of harmonics, pulses, and spikes in its waveform. This 10uF capacitor, along with a high-current series inductance, act together as a low-pass filter to remove this "hash" from the PWM-modulated AC generated by the full-bridge switching circuit.

This capacitor must continuously withstand high-frequency ripple currents of 10 to 20A. This is a lot of ripple current for a single capacitor. It requires a "pulse-rated" capacitor, or in larger inverters, a ladder of capacitors connected in parallel to provide sufficient ripple current handling capability. For example, instead of replacing it with another 10uF 400V capacitor, a small PC board or perfboard could be added, holding 10 pieces of 1uF 400V capacitor connected in parallel. Alternately, the replacement could be 4 pieces of 2.2uF connected in parallel, or 2 pieces of 4.7uF connected in parallel. The goal is to share the ripple current among multiple capacitors.

The replacement capacitor (or capacitors) must be selected for having a "pulse rating" in their specs. This is a somewhat difficult spec to find. Some manufacturers claim "pulse rated" without publishing the actual "maximum ripple current" or "peak pulse current" (these parameters should be stated in Amps).

For reference, most back-up UPS power supplies made within the past 5 years have "2 step" conversion from battery to 120V AC:

Older back-up power supplies contain a large heavy 60Hz transformer. It is almost always bolted directly to the case of the UPS. The wire connections to its 12V primary side may be very thick wire with "bolt and nut" connections. The power rating of this transformer must be equal to the power rating of the UPS. Often this transformer weighs nearly as much as the batteries. This type of DC-AC inverter operates as a single stage, directly stepping up a 12V DC switched input into a 120V (or 240V) AC output. The 12V primary of this transformer is center-tapped. The center tap is connected directly to the +12V or +24V DC input rail coming from the battery (or batteries). Each outside terminal of the primary is attached to the drain terminal a high power MOSFET switching transistor. There are 2 such "switch" MOSFETs, one for each "side" of the primary. In larger units  each "side" may be comprised of 2, 4, or 8 individual MOSFETs connected in parallel.  The entire circuit operates much like a "push-pull" transformer-coupled audio amplifier output stage, except that it only operates at 60Hz (or 50Hz) and its output signal resembles a "square wave." In other words, the 2 switching transistor are alternately turned on to drive the outside terminals of the center-tapped inverter transformer primary.

UPS units that are not quite so old often additionally apply PWM (pulse-width modulation) to the inverter switching transistors. This enables them to generate a (somewhat) sinusoidal output signal rather than a square wave. Units with PWM have a filter network at their inverter output consisting of a series inductor and a parallel capacitor. As with newer high-frequency switching 2-stage UPS units, this filter capacitor often takes the form of a 10uF 400V polyester film capacitor. This capacitor is subjected to high-frequency ripple current from the PWM, although the peak current is lower than a newer 2-step high-frequency switching UPS design. The peak high-frequency current in these older units with the large transformer is lower because the large transformer itself has substantial self-inductance. Even so, this filter capacitor (if it needs replacement) should still be "pulse rated," similar to what is required for a 2-step high-frequency switching UPS.

I have several UPS units on my workbench right now. One is the newer 2-stage design. The others are much older units with large (and heavy) 60Hz inverter transformers. I'll post more info here after I finish them up. None show any signs of physical damage to their output filter capacitors. That said, I will  check these capacitors for correct uF. Sometimes this type of capacitor fails gradually, with the measured uF getting lower than the labeled value. But when a 10uF capacitor still measures 10uF (and shows no signs of overheating or other damage) then it is probably still good and there is no need to replace it.
 



     

Title: Re: APC SmartUPS 1000 failure (FIXED)
Post by: Lorenzo_1 on July 23, 2020, 07:55:40 am
That's a lot of really useful information for future reference.  I ordered a couple of replacement caps Illinois Capacitor 106PHC400K - they're rated 12A ripple current and 6.3mOhm ESR at 100kHz but no reference to 'pulse rated' in datasheet.  Does say they're suited for SMPS applications.  The cap I patched in was from a dead donor unit and only measured 7.5uF on test but figured it was enough to tell me if the unit would come back to life. The new ones come up smack on 10uF.  Looks like these things might go badly off spec (+/-10%) over time. 

Notable that the PCB is this UPS is pretty thick and I getting the old cap desoldered from the donor unit was hard going.  In the end I just clipped off the donor unit cap leads (which ended up too short for refitting) and patched the donor cap onto clipped off leads on the good unit.  I'll have another (careful) go at removing the old leads and refitting the new cap properly this time round. Not always easy to decide whether to just leave well alone and run with the 7.5uF donor cap or bite the bullet and install the new one. 'Leave well alone' is a good motto but I'm thinking I'm just buying another failure down the track if I don't renew it.

Agree with some of the sentiments about APC - they seem very rigid on not releasing schematics etc.  Not sure if they're are any quality UPS suppliers who do.   Last time I called APC about my dead UPS best they could offer IIRC was a $50 trade-in on a new unit.  Not exactly a bargain. I think I'll shop around before replacing this one.  There do seem to be a few APC schematics out there if you really hunt for them.
Title: Re: APC SmartUPS 1000 failure (FIXED)
Post by: elecdonia on July 23, 2020, 05:17:13 pm
I discovered a series of “teardown” articles at Tom’s hardware guide. If you go to the Tom’s hardware guide and search for “APC UPS teardown” it will bring up these articles. Although the articles don’t contain schematics, I found a large amount of useful information in them.

I also found some APC schematics in .pdf format with extensive Google searches. They are mostly for older models (1990-2010) but still, every little bit of technical info is helpful.

I have a couple of APC UPS units on my workbench at the moment.

One is an early 2000’s vintage model with 1100VA rated output and two 12V 12Ah batteries. It contains a very large and heavy iron-core inverter transformer. Interestingly, it has a tubular yellow 10uF 400V film capacitor across the inverter output. This capacitor looks like yours except in my case it hasn’t exploded and it tests good. Note that my UPS has 120V output voltage rather than 240V.

The other APC UPS unit on my bench is a newer “2 stage” 650VA unit. Its 12V DC-DC step-up section had shorted MOSFET switching transistors. I’m awaiting the replacement transistors to arrive. I suspect this might be its only fault.

I’ll post more info on these two units as I work on them.

Title: Re: APC SmartUPS 1000 failure (FIXED)
Post by: Lorenzo_1 on July 24, 2020, 02:22:28 am
I'll track that site down - thks.  Did a permanent replacement of faulty cap last night and all good. The one I temporarily patched in from my same-model donor unit was a 931C4W10 - 'K - F 10uf +/-10% 400VDC 85C. Couldn't find a datasheet. Bit surprised it wasn't higher temp rated.  Interestly, I noticed that one of the relays on this supposedly New/Old Stock UPS looks to have been replaced.  So perhaps it was in fact a repaired unit.  Anyway, got it for a good price so no complaints.
Title: Re: APC SmartUPS 1000 failure (FIXED)
Post by: TheMG on July 24, 2020, 02:32:05 am
Bit surprised it wasn't higher temp rated.

With electrolytic capacitors, the operating temperature has a direct effect on the longevity, and higher temperature rated ones tend to last longer.

Not the case with film capacitors. They don't degrade in the same way and temperature is much less of a factor.
Title: Re: APC SmartUPS 1000 failure (FIXED)
Post by: elecdonia on July 25, 2020, 03:19:29 am
I just finished restoring a late 1990's APC UPS, model BP1100.

My unit has the large 10uF tubular film capacitor connected across its 120V output.  This capacitor is physically similar in size and shape to the failed capacitor described in the first post of this thread. But in my unit this 10uF capacitor is still perfectly good.

However, I found nearly every ***small*** electrolytic capacitor inside this UPS had excessively high ESR, some measuring >50 ohms ESR. In comparison the new replacement capacitors have ESR  < 1 ohm.

The majority of these small electrolytic capacitors in the BP1100 model are 22uF, with either a 16V or 25V rating. There is also one 33uF and one 100uF capacitor in this BP1100 model.

People often focus on physically large electrolytic capacitors failing. But my experience is nearly the opposite: I often find the tiniest electrolytics on the PC board are the ones that develop very high ESR. I suspect this occurs because they contain only one or two drops of electrolyte solution. If/when that leaks out or evaporates, then the capacitor's ESR increases by 10x to 50x over the original value.

Conclusion: If you are repairing a UPS which is more than 10 years old, then I strongly recommend replacing ***every*** small electrolytic on its main PC board. To avoid mistakes I replace one capacitor at a time, carefully checking the + and - polarity.

Title: Re: APC SmartUPS 1000 failure (FIXED)
Post by: elecdonia on July 25, 2020, 03:29:27 am
The cap I patched in was from a dead donor unit and only measured 7.5uF on test but figured it was enough to tell me if the unit would come back to life. The new ones come up smack on 10uF.  Looks like these things might go badly off spec (+/-10%) over time.
This is a "metallized film" capacitor. This type tends to gradually lose some uF value as it ages. This is part of their internal "self healing" feature: Momentary internal arc-overs vaporize a bit of the metallized film, leaving a hole in both the dielectric film and the metallized electrode surface on either side of the dielectric. The capacitor doesn't short out, but as the holes grow larger, the uF decreases.

I learned this from an HVAC technician who was trained to measure the uF of every "motor run capacitor" inside my furnace and outside air conditioner unit. He said that metallized film capacitors gradually decrease in uF as they age. His rule was that capacitors measuring 20% lower than their labeled uF should always be replaced. He said that if these weak capacitors are left in place they will eventually fail catastrophically.
Title: Re: APC SmartUPS 1000 failure (FIXED)
Post by: coromonadalix on July 25, 2020, 03:31:00 am
my apc  had the same small capacitors problems, it overcharged the 2 batteries, i saw the apc case badly deformed, maybe a day before blowing them up ???
Title: Re: APC SmartUPS 1000 failure (FIXED)
Post by: elecdonia on July 25, 2020, 10:11:53 pm
Quote
However, I found nearly every ***small*** electrolytic capacitor inside this UPS had excessively high ESR, some measuring >50 ohms ESR. In comparison the new replacement capacitors have ESR  < 1 ohm. The majority of these small electrolytic capacitors in the BP1100 model are 22uF, with either a 16V or 25V rating. There is also one 33uF and one 100uF capacitor in this BP1100 model.

Today I recapped another APC BP1100 UPS.

This unit was in operational condition until its batteries failed from old age. Then it was set aside after its owner decided to replace it with a new rack-mounted UPS. So I got the old UPS for free.

When I checked it out I discovered several tiny electrolytic capacitors which have high ESR. As with the other APC BP1100 I posted about, I decided to replace every small electrolytic capacitor on its PC board.

I don't see any point in putting an expensive set of new batteries into an older UPS which might "cook" the new batteries because it contains a bunch of failed electrolytic capacitors which can be easily replaced. With the capacitors replaced I expect this BP1100 will work correctly for a long time.

There are seven 22uF 16V capacitors inside this BP1100 model. Locations on the PC board are C34, C35, C13, C37, C53, C54, and C80. The old capacitors had ESR ranging from 6 to 24 ohms. The new replacement capacitors all have ESR < 1 ohm. Interestingly, all of the 22uF capacitors in this BP1100 were 16V, but in the other BP1100 unit I recapped last week 3 of its 22uF capacitors were 25V.  The in-circuit applied voltage for all of them is <12V, so I don't think APC initially put in the wrong capacitors at the factory. Also it's OK to use replacement electrolytics with slightly higher voltage ratings than the originals.

There are only 2 other small electrolytics on the BP1100 PC board: C40 (33uF 35V) and C48 (100uF 6.3V). These had ESR in the 1.5 to 2 ohm range, which is probably OK, but I went ahead and replaced them anyway.

I'll be testing this unit soon and will post again about its performance. Although I currently have two units of the APC BP1100, I only have one set of good batteries to test with at the moment. 

These APC BP1100 units contain a pair of 12V 12Ah batteries. I buy SLA lead-acid batteries from Interstate Battery. These aren't the cheapest batteries around, but there is a local Interstate Battery store where I live so I don't have to pay anything extra for shipping. These 12V 12Ah batteries cost about $50 each at Interstate Battery. I've always been quite satisfied with Interstate Battery as a brand, but I'm open to suggestions for other battery brands which have worked well for EEVblog members. That said, I don't think a 12V 12Ah battery which sells for only $25 can be as good.

 
Title: Re: APC SmartUPS 1000 failure (FIXED)
Post by: james_s on July 25, 2020, 11:56:29 pm
I've heard that there are only a few different companies that actually make SLA batteries, a lot of brands don't supply proper datasheets though so it's hard to know what you're getting. Power Sonic and SigmasTek at least do provide good datasheets. As far as buying locally, I've found you do pay shipping, you just pay shipping to the store instead of to your home. For example I can buy a Power Sonic PS-1290 online for around $20 shipped, or I can buy the exact same PS-1290 battery locally at Home Depot for $28.
Title: Re: APC SmartUPS 1000 failure (FIXED)
Post by: Dacke on July 26, 2020, 01:44:21 am
I bought the last set of 12V 12AH SLA batteries for my UPS from a Batteries Plus store a couple miles from me,  I believe it was these -  https://www.batteriesplus.com/productdetails/battery/sla-sealed-lead-acid/12/sla12=12f2 (https://www.batteriesplus.com/productdetails/battery/sla-sealed-lead-acid/12/sla12=12f2)

They are about 3 1/2 years old and still going fine.  I used to buy the $25 generic ones from Amazon but I got tired of having to replace them every 18 - 24 months.
Title: Re: APC SmartUPS 1000 failure (FIXED)
Post by: james_s on July 26, 2020, 02:10:11 am
I don't know who actually manufactures the Duracell branded SLA batteries but the datasheet doesn't really tell you anything. Keep in mind that is listed as a general purpose battery for emergency lights and is not specifically stated as a high rate battery optimized for UPS service. In some cases there is no real difference between general purpose and high rate batteries but that may not always be the case. It's possible that the ones you bought from Amazon were designed for low current emergency lighting service, it's also possible they were just crappy batteries.
Title: Re: APC SmartUPS 1000 failure (FIXED)
Post by: elecdonia on July 27, 2020, 08:00:26 pm
I don't know who actually manufactures the Duracell branded SLA batteries but the datasheet doesn't really tell you anything. Keep in mind that is listed as a general purpose battery for emergency lights and is not specifically stated as a high rate battery optimized for UPS service. In some cases there is no real difference between general purpose and high rate batteries but that may not always be the case. It's possible that the ones you bought from Amazon were designed for low current emergency lighting service, it's also possible they were just crappy batteries.
My research indicates that the 12V 12Ah size SLA battery doesn't come in a specific "high-rate" version. It seems that it's mostly the smaller SLA batteries, such as 12V 6Ah, where there are different versions of the same physical size battery to serve different applications.

I looked up pricing for the Durcacell brand 12V 12Ah SLA. It is $45 each, which isn't far from the $50 I paid for the Interstate Battery 12V 12Ah SLA.

The cheap 12V 12aH batteries mostly come from Amazon sellers, with prices ranging from $25 to $30 each. If anyone has tried these in a UPS and obtained good results (with long life) please post the details. At this point in time I think I will stay with the major "name brands" which all appear to be priced between $40 and $50 each for the 12V 12Ah size SLA.

-Elecdonia




Title: Re: APC SmartUPS 1000 failure (FIXED)
Post by: Lorenzo_1 on July 27, 2020, 11:25:03 pm
I used Yuasa NP series batteries WITH GOOD SUCCESS. I think the current set has been in about 4 years and still test OK.
Title: Re: APC SmartUPS 1000 failure (FIXED)
Post by: Lorenzo_1 on July 27, 2020, 11:27:26 pm
Sorry - didn’t mean to shout :palm: using phone for posts a mistake.
Title: Re: APC SmartUPS 1000 failure (FIXED)
Post by: james_s on July 28, 2020, 03:48:37 am
I've had Yuasa batteries that were good too, they used to be very common but now that you mention it I haven't seen one in a while.
Title: Re: APC SmartUPS 1000 failure (FIXED)
Post by: elecdonia on July 29, 2020, 06:41:58 pm
I've had Yuasa batteries that were good too, they used to be very common but now that you mention it I haven't seen one in a while.
The Yuasa brand name is still around. I've seen them for sale from several sites. They tend to be priced a bit higher than generic SLA batteries. For example, the 12V 12Ah size I need for my APC Back-UPS BP1100 units is roughly $50 each for the Yuasa brand.

What I'm curious about is whether any of the cheaper generic SLA batteries are good enough to use?

An Amazon seller is offering a 12V 12Ah battery for $29.15 including shipping. The brand is "ExpertPower."  Amazon users rate it at 4.4 out of 5 stars with 1,218 users reviewing it. So I'm thinking this might be a decent quality battery. It appears these 12V 12Ah SLA batteries are very popular for "mobility scooters." So they are sold in much larger quantities than some of the other sizes.

-E
Title: Re: APC SmartUPS 1000 failure (FIXED)
Post by: Lorenzo_1 on July 29, 2020, 10:29:02 pm
Don’t know about SLA batteries but I recently had to return generic Chinese12v automotive lead acid battery that failed after just 18 months with dead cell. Retailer said he stopped selling them b/c too many warranty returns. I keep away from them now. FWIW.
Title: Re: APC SmartUPS 1000 failure (FIXED)
Post by: james_s on July 30, 2020, 07:01:23 am
I've recently bought some from Power Sonic and SigmasTek, both were chosen because they provide decent datasheets that show the batteries at least claim to meet the specs I need. Ask me in a few years and I'll be able to tell you if they've held up.
Title: Re: APC SmartUPS 1000 failure (FIXED)
Post by: GreyWoolfe on July 30, 2020, 01:29:30 pm
Back in 2004, my company deployed a couple of hundred SmartUPS 1500's and they all came with Yuasa batteries.  IIRC, they lasted 4 years before we started seeing some failures.  At that point the decision was made to proactively replace all the other batteries before it became an issue.  We were able to get Yuasas for that go around but when it came time to replace them again, Batteries Plus didn't carry them anymore and we ended up with Duracell batteries I believe and they also performed well.  By the way, we used Batteries Plus as the company has a national account.