EEVblog Electronics Community Forum
Electronics => Projects, Designs, and Technical Stuff => Topic started by: dmg on December 20, 2017, 05:48:57 pm
-
We have just bought a new TTI CPX200D power supply for our lab, and it arrived today. We needed a power supply with remote sense capability to better power an RF board. The supply arrived this morning and we tested it for voltage and current and with some loads and everything looked OK, so this afternoon we proceeded to test the RF board. We had powered up the RF board previously with another supply without problems, but there were issues with voltage drop in the wiring.
We setup everything following both the supply manual and the RF board manual. The RF board gets its power from a connector, and outputs the two power sense lines, connected directly to the supply voltage and ground at the power connector. We twisted the sense wires together and connected them to the SENSE inputs of the supply, and connected the power wires to the output terminals. We switched the supply to remote sense, triple checked the connections polarity, adjusted voltage and current limit (the RF board is powered at 1.8V 1A, but the manufacturer recommends starting it with a 50mA current limit just in case and then raising it, so we set the current limit to 50mA) and proceeded to connect the power connector to the RF board, all of this with the supply output DISABLED.
Just when the power connector was starting to mate with its socket in the RF board, a tantalium capacitor connected across the power rail just at the RF board power connector blew up and started burning, and the board is now ruined. The RF board was worth 10000€. There was no other energy source for the RF board, no other active thing was connected to it. No instrument was connected either. Nothing. It has a single supply rail at 1.8V, we had succesfully powered it up before several times and, again, the output was DISABLED. With no other energy source available for the RF board, the energy responsible for blowing the capacitor (220 uF 16V tantalium cap) had to come from the power supply somehow. The thing is, we also had a multimeter connected accross the power lines monitoring the output voltage and it said 0V. Sense and power rails get tied together when the power connector is mated with the RF board, and these sense lines were the only non-monitored part we had. We checked afterwards that with the output DISABLED the sense lines remain at 0V and that there is no transient nor anything when power and sense lines get tied together.
Do you have any theory about how this happened?
EDIT:
Here's an update. During this time we tried simulating the board with a dummy load, consisting of an electrolytic capacitor with similar specs to those of the tantalum that blew up (electrolytics can tolerate much more without blowing up with toxic smoke) and found some interesting things.
If you proceed exactly as we proceeded when the incident happened (power supply channel turned off), nothing happens. The output stays at 0V, no transient, no nothing. No matter how you connect the supply to the dummy load, nothing happens. If the capacitor is pre-charged it just discharges with a nice exponential curve (so the output has confirmed bleeding resistors)
If you then turn on the supply with the dummy load already connected, the output just rises nicely to 1.8V, no overshoots or weird stuff.
BUT, if you turn on the power supply channel with the dummy load disconnected and you then hot-plug the dummy load, in some cases there's a several 10's of volts transient (>40V), almost 3 seconds long, and then the output stays at 1.8V. That scenario would have explained our issue... if it wasn't for the fact that the PSU supply channel was disabled. We triple-checked that it was indeed disabled before plugging anything. Both my colleague and I are 100% sure that it was off.
We'll keep investigating... (the manufacturer hasn't answered yet, we'll try new communication channels).
-
Have you measured the output of the PSU to the earth and/or to the GND of the RF board (in case it is connected to something else)?
I had a case a bad insulation made a bridge of the secondary to earth within my PSU and nearly killed my scope that way.
-
Have you measured the output of the PSU to the earth and/or to the GND of the RF board (in case it is connected to something else)?
I had a case a bad insulation made a bridge of the secondary to earth within my PSU and nearly killed my scope that way.
Yep, we tested supply negative and positive to ground isolation, It's properly isolated. No other connection to the RF board, just power. We're just puzzled.
EDIT: At least the cap was nice enough to spew the burning tantalium sparks sideways and not towards my face. For now we're assuming that if there was enough energy to blow a tantalium cap, there was enough energy to blow the main expensive IC at the board (no regulators in between... it's an evaluation board for a chip and it's made that way). We'll try to replace the cap and, if there's no hole in the PCB below it, try again and see if the IC is fried.
-
A tantalum cap right at the input of a high current (e.g. 1 A) external supply is not such a good idea. Some tantalum caps don't like too high current spikes. An expensive board with external supply should be protected against supply transients - for an RF board there might need to be RF filtering anyway - this could have saved the cap.
For a supply with external sense lines the sense lines , usually the sense wire should be twisted with the respective power wire and not the two sense wired together. Supplies have a very hard time to compensate external inductance and thus can get unstable worst case. So if the sense wires are twisted together, the power lines should be twisted as well - still not the preferred configuration.
Some voltage regulators might react a little odd with the sense lines not connected and thus possibly with transient voltages outside there operating rage. A good supply should be protected against this, but sometimes protection is not enough against ESD events on some inputs. For some strange reason a fast common mode change for the sense wires might get through, even with the output disabled.
It might be interesting to see, how high the input impedance is with the sense lines - if very high impedance, they might catch capacitive coupled line voltage and thus rather high amplitudes. So connecting sense and power part may not behave the same every time.
-
A tantalum cap right at the input of a high current (e.g. 1 A) external supply is not such a good idea. Some tantalum caps don't like too high current spikes. An expensive board with external supply should be protected against supply transients - for an RF board there might need to be RF filtering anyway - this could have saved the cap.
For a supply with external sense lines the sense lines , usually the sense wire should be twisted with the respective power wire and not the two sense wired together. Supplies have a very hard time to compensate external inductance and thus can get unstable worst case. So if the sense wires are twisted together, the power lines should be twisted as well - still not the preferred configuration.
Some voltage regulators might react a little odd with the sense lines not connected and thus possibly with transient voltages outside there operating rage. A good supply should be protected against this, but sometimes protection is not enough against ESD events on some inputs. For some strange reason a fast common mode change for the sense wires might get through, even with the output disabled.
It might be interesting to see, how high the input impedance is with the sense lines - if very high impedance, they might catch capacitive coupled line voltage and thus rather high amplitudes. So connecting sense and power part may not behave the same every time.
The design is not ours, it's the official evaluation board from the manufacturer of a pre-release RF IC. They recommend the power supply with sense lines stuff. If it was me who designed the thing, I'd provide the power locally through a regulator but...
You're right about the twisting, we did it as you say, I explained it wrong before. Tantalium smoke inhaling... you know.
But apart from all of this, the supply was disabled. I mean, output OFF. Nothing should blow with output OFF.
-
I wonder if when the supply output is disabled it is truly disconnected or is it connected but only set to deliver 0 V? In the latter case, could quickly connecting a capacitor to the sense leads trick the supply into generating an output pulse?
Best o' luck!
-
Something we always do on remote-sense supplies is put some resistors between +out and +sense, and -OUT and -sense. This prevents the supply from running away if the sense circuit is not actually connected. I usually use about 100 Ohms for these, but it depends on the load the remote sense input imposes.
Not sure if your supply has these, you could just use a DVM to see if they have such in their circuit. If NOT, then I'd definitely add this.
Jon
-
Something we always do on remote-sense supplies is put some resistors between +out and +sense, and -OUT and -sense. This prevents the supply from running away if the sense circuit is not actually connected. I usually use about 100 Ohms for these, but it depends on the load the remote sense input imposes.
Not sure if your supply has these, you could just use a DVM to see if they have such in their circuit. If NOT, then I'd definitely add this.
Jon
-
I had a lovely gotcha like that, lab power supply turned out to have a honking great cap across the output.
Had been doing something with 12V or so, switched off, reset the output voltage to 2.5V or so, plugged in the load....
Swearing ensued.
Regards, Dan.
-
Owch!
I'd be tempted to try something.... Get a cap the same as on your board (need a new one for the board, anyway) and a 2r resistor in parallel. And maybe a connector, to make a passive version of your board's load...
Put an oscilloscope across it, set to trigger on any voltage rise above 1v..
Do your thing with the PSU, then plug it in.
See if you can see anything at all that might kill your board.
-
Something we always do on remote-sense supplies is put some resistors between +out and +sense, and -OUT and -sense. This prevents the supply from running away if the sense circuit is not actually connected. I usually use about 100 Ohms for these, but it depends on the load the remote sense input imposes.
Not sure if your supply has these, you could just use a DVM to see if they have such in their circuit. If NOT, then I'd definitely add this.
Jon
This particular supply has a limited 0.5V compensation range for the SENSE lines
I had a lovely gotcha like that, lab power supply turned out to have a honking great cap across the output.
Had been doing something with 12V or so, switched off, reset the output voltage to 2.5V or so, plugged in the load....
Swearing ensued.
Regards, Dan.
That's strange, even my personal 100€ PSU has bleeding resistors across the output caps to avoid that... In this case, I was explaining my colleague how the supply works just before that, and the last thing I did was toggling the PSU while set to 1.8V, then I locked the dials and, with the supply switched off, I proceeded to connect power, and boom.
Owch!
I'd be tempted to try something.... Get a cap the same as on your board (need a new one for the board, anyway) and a 2r resistor in parallel. And maybe a connector, to make a passive version of your board's load...
Put an oscilloscope across it, set to trigger on any voltage rise above 1v..
Do your thing with the PSU, then plug it in.
See if you can see anything at all that might kill your board.
I'll definitelly do that, but inside a fume hood and with safety goggles. I'm lucky to have only experienced burnt tantalium scent about 5 times in my life and sincerely I don't really want to get that privilege again. I love when the damn thing gets exposed to air after blowing off and initiates a self-sustaining fire even if you cutoff the power, and you can only enjoy while it drills a hole through your PCB. Nasty stuff.. I try to avoid them like the plague in my designs in favour of ceramics/polymer, but sometimes they're the best choice....
-
I had a lovely gotcha like that, lab power supply turned out to have a honking great cap across the output.
Had been doing something with 12V or so, switched off, reset the output voltage to 2.5V or so, plugged in the load....
Swearing ensued.
Regards, Dan.
I have to test that one with my own supplies, to avoid a surprise.
-
Keeping in mind that the output was disabled when the damage occurred, I'm curious about what this means, "Sense and power rails get tied together when the power connector is mated with the RF board,"?
-
I would guess the power supply disables the output by setting voltage to zero. The disconnected sense leads allowed the terminals to float from leakage to some voltage, as there was no feedback from any sensing to downprogram the voltage.
-
Something we always do on remote-sense supplies is put some resistors between +out and +sense, and -OUT and -sense.
Do you connect the resistors right at the PS output terminals ?
-
I would guess the power supply disables the output by setting voltage to zero. The disconnected sense leads allowed the terminals to float from leakage to some voltage, as there was no feedback from any sensing to downprogram the voltage.
But according to the manual, the correction range through sense is 0.5V, that should mean the output should have stayed to within +-0.5V during this event, and that's not enough to blow a tantalium cap.
I mean, I don't expect to have to take these sort of precautions when connecting a disabled 900€ PSU to something. In fact, how else could I protect the circuit against startup transients?
Sure thing, the RF board is poorly designed (no local decoupling near the chip, too much imput capacitance after too long wires (they provide the power wiring with the kit), no local regulator, having to use a supply with sense capability to compensate for voltage drop during transients as there's no internal regulator, no TVS, no anti-parallel diode at the input...) but still... a disabled PSU should be disabled.
Keeping in mind that the output was disabled when the damage occurred, I'm curious about what this means, "Sense and power rails get tied together when the power connector is mated with the RF board,"?
That means that the power wire harness has 4 independent wires, and the mating on-board connector has two pairs of pins connected together, one to the supply rail, the other to ground. One pin of each pair is meant to supply voltage, the other to sense the voltage right at the load.
Let me take a wild guess: RFSoC?
We work a lot with xilinx parts and have quite a lot of Xilinx eval boards from Virtex 2 Pro era to modern Zynq stuff. Haven't tested RFSoC's yet, but I'm pretty sure that Xilinx would never design an eval board with such design flaws in the supply section. Moreso when their devices are notorious for their complicated power sequencing scheme, that I myself have suffered already.
It's a way more obscure IC for a niche application, can't disclose sorry.
-
Since this is a brand-new PSU, did you contact the manufacturer for comment?
-
Could it be a fault in the PSU that lead to AC voltage on the output? You'd probably have missed that when you checked with the DMM.
-
I am no expert, but one random thought I've had is perhaps the tantalum was charged when you connected it and the power supply suddenly shorted it causing a large discharge current? (It perhaps had been charged when the board was previously tested.)
I've no idea if this is a possibility but tantulums don't like large currents and I guess this also includes a discharge current.
-
Since this is a brand-new PSU, did you contact the manufacturer for comment?
Yes, waiting for their answer!
Could it be a fault in the PSU that lead to AC voltage on the output? You'd probably have missed that when you checked with the DMM.
Didn't check that, but I'm pretty sure I touched the output leads when I was testing it and didn't get a shock. Will test anyway, but if that's the fault then well...
I am no expert, but one random thought I've had is perhaps the tantalum was charged when you connected it and the power supply suddenly shorted it causing a large discharge current? (It perhaps had been charged when the board was previously tested.)
I've no idea if this is a possibility but tantulums don't like large currents and I guess this also includes a discharge current.
Not likely, the board had been sitting inside its box for almost a week, any possible charge should have leaked already. Also, we checked today and the main IC on board is totally fried, so there was definitelly some spike or something.
The good news is that the tantalum didn't burn a hole through the board, the PCB is undamaged. We'll try to swap the main IC and repair the board.
-
However it happened, rather than high voltage, maybe it was reverse polarity. This would more likely explain the tantalum capacitor failing.
-
Just found in the power supply specs "Reverse protection by diode clamp for currents to 3A". So reverse polarity is unlikely.
-
Imagine this: The tantalum was charged when you connect the PSU. The PSU voltage servo circuit suddenly sees this voltage on its sense wires. It concludes that its output is wrong, and immediately attempts to correct the situation by applying the opposite voltage to bring things under control. Unfortunately it overshoots the zero volt point and ends up in negative territory (even for a short instant), which blows the tantalum capacitor (which is sensitive to reverse voltages).
You can test this by connecting a charged capacitor of about the same size to the supply and watch the party on a scope?
-
The PSU can't do that (apply negative voltage), it's not a 4Q unit.
-
Here's an update. During this time we tried simulating the board with a dummy load, consisting of an electrolytic capacitor with similar specs to those of the tantalum that blew up (electrolytics can tolerate much more without blowing up with toxic smoke) and found some interesting things.
If you proceed exactly as we proceeded when the incident happened (power supply channel turned off), nothing happens. The output stays at 0V, no transient, no nothing. No matter how you connect the supply to the dummy load, nothing happens. If the capacitor is pre-charged it just discharges with a nice exponential curve (so the output has confirmed bleeding resistors)
If you then turn on the supply with the dummy load already connected, the output just rises nicely to 1.8V, no overshoots or weird stuff.
BUT, if you turn on the power supply channel with the dummy load disconnected and you then hot-plug the dummy load, in some cases there's a several 10's of volts transient (>40V), almost 3 seconds long, and then the output stays at 1.8V. That scenario would have explained our issue... if it wasn't for the fact that the PSU supply channel was disabled. We triple-checked that it was indeed disabled before plugging anything. Both my colleague and I are 100% sure that it was off.
We'll keep investigating... (the manufacturer hasn't answered yet, we'll try new communication channels).
-
I have downloaded the service manual from the TTI CPX 200, I don't know if the shematics are the same as CPX 200D
https://elektrotanya.com/tti_cpx200_sm.pdf/download.html
1) it is a SMPS, not a linear power supply....not a good idea to try spare some hundreds of $ when you are testing $ 10.000.00 boards. Also no crowbar protection....Why did you not buy a Keyside power supply ?
2) Output is internally never disconnected from power converter, this is not safe for the circuit connected to the output.
3) Output circuit has very high value capacitors (because it is a SMPS !)...On the schematic, I see 2.200 + 470 + 470 = 3.140µF....That's crazy !!!!!
There are three 1.200R 2W resistors in parallel, to discharge the output capacitors....
But, what probably happened: for some reason, during adjust of current limit, these condensators has been charged to the maximum output voltage.
They don't discharge instantanely, you did not measure the output voltage before connecting the power supply to the board, the output condensators where not still fully discharged, there was still a voltage greater than 1.8V and you killed your board....
-
I have downloaded the service manual from the TTI CPX 200, I don't know if the shematics are the same as CPX 200D
https://elektrotanya.com/tti_cpx200_sm.pdf/download.html
1) it is a SMPS, not a linear power supply....not a good idea to try spare some hundreds of $ when you are testing $ 10.000.00 boards. Also no crowbar protection....Why did you not buy a Keyside power supply ?
2) Output is internally never disconnected from power converter, this is not safe for the circuit connected to the output.
3) Output circuit has very high value capacitors (because it is a SMPS !)...On the schematic, I see 2.200 + 470 + 470 = 3.140µF....That's crazy !!!!!
There are three 1.200R 2W resistors in parallel, to discharge the output capacitors....
But, what probably happened: for some reason, during adjust of current limit, these condensators has been charged to the maximum output voltage.
They don't discharge instantanely, you did not measure the output voltage before connecting the power supply to the board, the output condensators where not still fully discharged, there was still a voltage greater than 1.8V and you killed your board....
Ours is the CPX200D, different specs than the CPX200, but it looks like it's SMPS too. Didn't see that coming actually... my bet was that it was some sort of dual stage supply, with SMPS pre regulation and linear postregulation. Damn it, I should have RTFM better before choosing it.
The "input never disconnected from the power converter" stuff must be true for this model too, I don't hear any relay clicking when I disconnect the output.
As for the why we didn't get a Keysight, well, we do already have Agilent/HP old-ish supplies that work a charm and we already use them to power expensive stuff, in fact the first supply I used to power this board without problems is an Agilent. The thing is, none of our supplies had remote sense capability and we needed it for this board. TTi is a very reputable PSU brand and we decided to go for this one, first because it's hard to find supplies with remote sensing in other manufacturers and secondly because of the PowerFlex stuff.
We are going to repair the damaged board by replacing the expensive IC that got burned and we've also designed a "modchip" board that would be soldered to the expensive board. It contains a linear, ultra low noise regulator, reverse polarity protection, overvoltage protection and all that stuff. That way we can power the expensive board from say 2.5V and have proper local regulation at 1.8V.
-
Thanks for the update. Always good to know the rootcause so we can avoid making the same mistake! :-+
-
The "input never disconnected from the power converter" stuff must be true for this model too, I don't hear any relay clicking when I disconnect the output.
It's not necessary to have a relay, it could be mosfet switch, too.
Anyway, what you described is very strange, it's hard to imagine that design is so bad that max. voltage was present for 3 seconds on the output due to possibly control loop(s) failure. It looks more that you had a really bad luck with your unit, and hopefully TTi will let you know soon what could be a problem.
-
I do not think that we already know the root cause. Several shortcomings of the PS have been found, but the effect has not been definitely linked to one of them under the conditions described.
The assumption with the chargeup during current programming would be possible. Now it would be interesting if the OP could give us a timescale for the actions which he was describing so exactly. This might verify or invalidate the hypothesis.
-
I do not think that we already know the root cause. Several shortcomings of the PS have been found, but the effect has not been definitely linked to one of them under the conditions described.
The assumption with the chargeup during current programming would be possible. Now it would be interesting if the OP could give us a timescale for the actions which he was describing so exactly. This might verify or invalidate the hypothesis.
In addition to the defects and limitations of the design of this power supply, there are many other elements that confirm the explanation that I gave:
1) it was a new power supply, nobody was used to its use: a wrong maneuver could easily have been committed.
"We have just bought a new TTI CPX200D power supply for our lab, and it arrived today. We needed a power supply with remote sense capability to better power an RF board. The supply arrived this morning"
2) the settings were made without load. This make possible the internal 3.140µF to be charged at higher voltage than 1.8V.
"The RF board gets its power from a connector,"
3) The fact that the tantalum capacitor has started to burn proves that a voltage well above 1.8V has been applied to the input of the RF module and that, with enough energy to burn the capacitor .... This is not ESD or leakage current.
"Just when the power connector was starting to mate with its socket in the RF board, a tantalium capacitor connected across the power rail just at the RF board power connector blew up and started burning, and the board is now ruined. "
4) The output voltage of the power supply has not been checked with a multimeter before connecting the RF module. They relied only on the fact that the power was with its output "disabled"
"so we set the current limit to 50mA) and proceeded to connect the power connector to the RF board, all of this with the supply output DISABLED."
-
I do not think that we already know the root cause. Several shortcomings of the PS have been found, but the effect has not been definitely linked to one of them under the conditions described.
The assumption with the chargeup during current programming would be possible. Now it would be interesting if the OP could give us a timescale for the actions which he was describing so exactly. This might verify or invalidate the hypothesis.
In addition to the defects and limitations of the design of this power supply, there are many other elements that confirm the explanation that I gave:
1) it was a new power supply, nobody was used to its use: a wrong maneuver could easily have been committed.
"We have just bought a new TTI CPX200D power supply for our lab, and it arrived today. We needed a power supply with remote sense capability to better power an RF board. The supply arrived this morning"
2) the settings were made without load. This make possible the internal 3.140µF to be charged at higher voltage than 1.8V.
"The RF board gets its power from a connector,"
3) The fact that the tantalum capacitor has started to burn proves that a voltage well above 1.8V has been applied to the input of the RF module and that, with enough energy to burn the capacitor .... This is not ESD or leakage current.
"Just when the power connector was starting to mate with its socket in the RF board, a tantalium capacitor connected across the power rail just at the RF board power connector blew up and started burning, and the board is now ruined. "
4) The output voltage of the power supply has not been checked with a multimeter before connecting the RF module. They relied only on the fact that the power was with its output "disabled"
"so we set the current limit to 50mA) and proceeded to connect the power connector to the RF board, all of this with the supply output DISABLED."
About 1: Yes, it was new. I myself was familiar with TTi supplies from a proyect I worked on a while ago. We spent part of the morning playing with it after it arrived and we did some tests to check its functionality.
About 2: maybe I didn't explain myself well. What I meant is that the RF board has a power connector that mates with a connectorized wire harness also provided with the evaluation kit. That wire harness is what we connected to the power supply after twisting the wires. We did the settings without load, yes, but we turned the channel on and off after setting it. We also had a monitoring multimeter connected in parallel with the supply output, and we checked that it was outputting 1.8V when on, 0V when off.
About 3: yes, it was a high energy event. The tantalum cap was obliterated. Surely enough with tantalum caps you only need enough energy to heat the cap to ignition temperature and then it carries on by itself, but still... The cap was a 16V rated one.
About 4: yes, it was checked before, a multimeter was always present. We disabled the ouput channel before mating the power connector precisely to avoid hot-plug and the issues it carries. Unfortunately our multimeter can't log and it wasn't set to max hold or anything, so we don't know what peak voltage was applied to the board.
We don't yet know the root cause. We have found an scenario where the supply goes to 40V if you hot-plug a capacitive load with remote sense and the capacitor is discharged (if it's precharged to 1.8V or so it doesn't trigger the high voltage transient). But that scenario requires hot-plugging the load with the output channel enabled. It was disabled in our case, we're sure about that.
Now we need to see if it's possible to somehow trigger the transient with the supply channel disabled.
-
Now we need to see if it's possible to somehow trigger the transient with the supply channel disabled.
Yes, it is.... :scared:
If you look at the schematics, you will see that the output on/off switch controlled two separate functions : the "disabled" indication and the function itself.
There is absolutely no guarantee that when the disable function is displayed, the control is actually disabled.
It is quite a crap design, perhaps good for hobby but for sure not safe enough to use with $ 10.000 boards.
-
About 4: yes, it was checked before, a multimeter was always present. We disabled the ouput channel before mating the power connector precisely to avoid hot-plug and the issues it carries. Unfortunately our multimeter can't log and it wasn't set to max hold or anything, so we don't know what peak voltage was applied to the board.
You still have that supply, right? Test supply, check that scenario with other kind of load and scope as logger. If supply is to blame, then who knows.. maybe you can get 10k$ from TTI or their insurer :)
-
Looking at the CPX200 (10A) (schematic from link above ) and a CPX400a (20A) schematic and they are radically different.
The CPX 200 a dual switched flyback (No separate post regulator).
the cpx 400a is a dual switched forward converter + magamp for pre-regulation followed by linear output stage.
I also use a TTI supply the older Pl quad series with 'hard' switch disable and remote sensing, pure linear no pre-regulation. Been a very reliable supply so far.
A couple of possibilities to consider.
These supplies (and many others) use resistors (PTC in the cpx200) to guard against remote sense accidents. But as this previous post suggests (IMHO) I think resistors are inferior to diodes as guards as they offer no protection to loads in the 2 scenario's noted there . https://www.eevblog.com/forum/projects/korad-ka3003d-redesign-and-upgrade/msg1017611/#msg1017611 (https://www.eevblog.com/forum/projects/korad-ka3003d-redesign-and-upgrade/msg1017611/#msg1017611)
A 'guard diode' would always protect the load since it clamps the main output wire voltage to the remote sense wire voltage with a max difference of Vf ( i.e it protects the Load from excessive voltage in preference to protecting the sense wire from excessive current ).
The two scenario's :-
1 / If main wire has become disconnected (or similarly sense wires are connected first before main wires) then the main output terminal voltage 'Vmain' can rise up to to Vmax, this can happen because any current flowing through the guard resistors to the load causes a voltage drop 'Vrguard', so now Vmain = Vsense + Vrguard, any large output caps also get charged to Vmain. These will then get discharged into the load when the main wires are finally connected.
This may be occurring when your 'hot-plugging' which is connecting both the Vsense wires to the load a few mS before the main wire makes contact to the load. In those few mS Cout could charge to near Vmax.
The time constant of the CPX200's output caps and three parrallel output resistors is 3000u*400 = 1.2s thus for them discharge from 40V to ~ 2V = 3.6s , is this the 3 Seconds over voltage your referring to ?.
Another scenario that 'guard resistors' offer no protection against :-
2/ Mixing up polarity of sense wires . Again Vout will go up to Vmax for same reasons as 1 above and will remain there (if supply has an OVP trip that senses over voltage at the main wire (but not at the Vsense node) then this should activate and shut supply down) .
Is it possible to accidentally re-enable a bench supply .?
Well yes if it's one that disables it's output simply by setting Vset to 0V (a 'soft' disable) then consider it still in a normal 'on' state, it's just that it's output is now regulated to 0V . So as with any 'on' supply if we confuse it by applying a voltage to the Vsense terminal it will react. So a brief negative voltage applied to it's Vsense terminals (as would happen in a reverse polarity Vsense connection or Vsense connected to a negatively charged cap ) would cause the output voltage to rise sharply an attempt to maintain it's Vsense terminals at 0V.
But if that schematic linked in this thread is the correct one for your CXP200D I can't see how the output could be accidentally re-enabled by the method above since it looks like the output disable operates by shutting down the PWM oscillator that drives the flyback (shutdown by mcu).
A quick test :- with supply output disabled and in remote sense mode , charge a largish test cap (~-1V) and connect it to ONLY the Vsense leads (such that +Vsense and -Vsense will momentarily be reverse polarity ) whilst monitoring the main Vout terminals with scope.).
regards
-
Both my colleague and I are 100% sure that it was off.
Human memory is very unreliable, especially in accurately remembering a shocking/unexpected moments. You may verify something 3x and then accidentally press enable button while connecting wires.
-
Human memory is very unreliable, especially in accurately remembering a shocking/unexpected moments. You may verify something 3x and then accidentally press enable button while connecting wires.
The best part is our ability to lie to ourselves how reliable our memory is. Our brain is constantly trying to construct a coherent experience. Reality and truth don't seem to be a priority, only the appearance of it.
-
For me, it is clear that this power supply is intended for hobby use and is not at all suitable for a professional job with very expensive boards.
There are many errors or limitations in the project of this power that make it not safe for the circuits it feeds.
The big mistake in this accident is the choice of the brand and the model of the power supply ....
-
Watching - I like detective stories
-
The big mistake in this accident is the choice of the brand and the model of the power supply ....
As far as I'm concerned, the root cause is the manufacturer shipping a $10,000.00 demo board with no onboard regulation. No excuse for that.
It's in the nature of power supplies to misbehave destructively. The only safe assumption is that they will fail in the most unsafe way(s) possible. Power supplies are devilishly hard to get right, but they aren't sexy enough to attract the attention of talented, experienced EEs.
And you don't have to buy dodgy Chinese hacks to put your hardware at risk. People love the old HP/Harrison supplies, for example, but what happens when the wirewound pots they used develop a dirty spot? They fail by spiking to max voltage while you're adjusting them. Then there's the HP 6284A that blew up who-knows-how-many parts until someone (me) noticed that it was spiking on turn-off because of a race condition between the reference and output rails.
Even the IC manufacturers aren't immune. Who thought it was a good idea to build 78xx regulators without reverse shunt diodes....? Again, when they fail, they fail high. I could rant for days about this stuff...
-
Watching - I like detective stories
Enjoy it! But it seems more like a boring comedy...
Maybe the culprit is the material of the power supply feet! Are they of plastic or rubber? We have to analyze this!
From the initial description, it seems that was a user fault. They had to test a sensitive device, so they used a power supply with remote sense. But, instead of having fixed connections for this, they decided to connect it "on fly"! The consequences are known...
-
Both my colleague and I are 100% sure that it was off.
Human memory is very unreliable, especially in accurately remembering a shocking/unexpected moments. You may verify something 3x and then accidentally press enable button while connecting wires.
The sequence of events was as follows:
-Take power connector from the table, proceed to plug in
-Before connector is mated, BOOM.jpg
-I instinctively disconnect the wire and back off, my colleague backs off too.
-I go check the supply to turn it off, realize it is already off
-Without touching anything, we take picutres of everything
-Pictures show that the channel enable LED was off, and current meter shows set current (50mA) instead of 0, which is what it would show if the channel was ON
That's why I say we're pretty sure it was off. In any case, this was destined to happen. One time we would actually forget to turn the supply off or something like that and it would have blown off anyway.
We learned several lessons with this, the most important one is to setup a video camera and record everything the next time we try to power an expensive board xD. I'll keep updating.
Watching - I like detective stories
Enjoy it! But it seems more like a boring comedy...
Maybe the culprit is the material of the power supply feet! Are they of plastic or rubber? We have to analyze this!
From the initial description, it seems that was a user fault. They had to test a sensitive device, so they used a power supply with remote sense. But, instead of having fixed connections for this, they decided to connect it "on fly"! The consequences are known...
We connected it on the fly to avoid any startup surprise with the supply. I thought it was more reasonable to first power-up the supply, set it up properly and then, after it's properly setup, connect the board and finally enable the output channel.
I thought it was much safer to plug in the board with the supply in a known state and with its dials locked than to press the AC switch button with the board plugged in. Moreso in this case, where there's nothing between a 1000€ IC on a 10000€ board and the supply itself. We used the remote sense not because we liked the idea (it's idiotic not to have local regulation on-board) but because it was the manufacturer's recommended way of doing things. I didn't like the idea from the very beginning, also considering that the IC can draw current in pulses.
Now that the board is ruined, if we get a replacement or if we fix this board we'll hack it off either way and install a "modchip" that I designed with proper local regulation and protection.
-
The sequence of events was as follows:
-Take power connector from the table, proceed to plug in
-Before connector is mated, BOOM.jpg
-I instinctively disconnect the wire and back off, my colleague backs off too.
-I go check the supply to turn it off, realize it is already off
-Without touching anything, we take picutres of everything
-Pictures show that the channel enable LED was off, and current meter shows set current (50mA) instead of 0, which is what it would show if the channel was ON
That's why I say we're pretty sure it was off. In any case, this was destined to happen. One time we would actually forget to turn the supply off or something like that and it would have blown off anyway.
You could easily forget that you first disabled PSU before disconnecting the wire as a first measure. It's really hard to remember accurately when something like this happens. The only way to be sure would be only if you were taking video during the incident.
-
You could easily forget that you first disabled PSU before disconnecting the wire as a first measure. It's really hard to remember accurately when something like this happens. The only way to be sure would be only if you were taking video during the incident.
That seems like quite a stretch. If something blows up just as you're plugging it in, your attention isn't going to wander back to the power supply button. You're going to yank the connector out.
This is how I deal with $1000 eval boards with goofy power requirements and no supply conditioning:
(http://www.ke5fx.com/hmc661.jpg)
The box just contains a couple of 3-terminal regulators. Nothing fancy, but they can save the chip from most mistakes made by the board designer, the PSU, or me. :-BROKE
-
If something blows up just as you're plugging it in, your attention isn't going to wander back to the power supply button.
When you plug the wire and 10k board starts burning, the first thing you do is start panicking and loose attention to what you do.
-
If something blows up just as you're plugging it in, your attention isn't going to wander back to the power supply button.
When you plug the wire and 10k board starts burning, the first thing you do is start panicking and loose attention to what you do.
Even if your hands are on the plug at the time? Sorry, not buying it.
-
I think we already established that this is a pretty crappy supply.
For one, it can only regulate up, not down. If it is disable it will still rely on the internal bleed resistors to lower the output voltage.
WRT the PWM being disabled by the MCU: Even if the MCU could do this, we don't know if it's actually implemented that way.
Maybe they really just set the VSET voltage to zero and hope for the best.
If you look at the voltage amp in the schematics, you will notice it's a pure integrator. If the Vsense lines are disconnected from the output lines then there is no limit.
You should measure the resistance between The pos output and the pos sense and the neg output and the neg sense to see if the PTCs (why are they using PTCs?) are indeed there.
If there is an open, then your output voltage could be anywhere no matter what it is set to.
You could also test this directly by measuring the output voltage and then introduce a voltage difference on the sense terminal.
As others have said, I would definitely get rid of this supply. It's wrong in so many ways...
-
I think we already established that this is a pretty crappy supply.
For one, it can only regulate up, not down. If it is disable it will still rely on the internal bleed resistors to lower the output voltage.
WRT the PWM being disabled by the MCU: Even if the MCU could do this, we don't know if it's actually implemented that way.
Maybe they really just set the VSET voltage to zero and hope for the best.
If you look at the voltage amp in the schematics, you will notice it's a pure integrator. If the Vsense lines are disconnected from the output lines then there is no limit.
You should measure the resistance between The pos output and the pos sense and the neg output and the neg sense to see if the PTCs (why are they using PTCs?) are indeed there.
If there is an open, then your output voltage could be anywhere no matter what it is set to.
You could also test this directly by measuring the output voltage and then introduce a voltage difference on the sense terminal.
As others have said, I would definitely get rid of this supply. It's wrong in so many ways...
It's already established that you did not check the schematic of CPX200D, not even CPX200 which is completely different supply, otherwise you won't write what's in bold.
-
As of the safest way using TTI PSUs with remote sense. Disable output, set sense switch to local, connect all wires, set switch to remote, lastly press OUTPUT to enable.
-
I think we already established that this is a pretty crappy supply.
For one, it can only regulate up, not down. If it is disable it will still rely on the internal bleed resistors to lower the output voltage.
A design it shares with a very large majority of lab power supplies.
-
It's already established that you did not check the schematic of CPX200D, not even CPX200 which is completely different supply, otherwise you won't write what's in bold.
[/quote]
Would you have a link to the correct schematics?
Thanks!
-
Ours is the CPX200D, different specs than the CPX200, but it looks like it's SMPS too. Didn't see that coming actually... my bet was that it was some sort of dual stage supply, with SMPS pre regulation and linear postregulation. Damn it, I should have RTFM better before choosing it.
Yep, you have chosen a lot of current and voltage for relatively low price. Sure it's not the best TTi PSU to use with extremely expensive and easy to fry circuits. But I still don't think that it was an issue with PSU, rather a user error. Especially considering that you were connecting wires with remote sense switch in enabled position.
-
Would you have a link to the correct schematics?
Thanks!
For CPX200 there is a link in a some post above, it certainly won't misbehave with sense wires disconnected as you wrote. But it could do so if you mishandle sense wires.
For CPX-200D i guess you would need try asking TTi.
-
Would you have a link to the correct schematics?
Thanks!
For CPX200 there is a link in a some post above, it certainly won't misbehave with sense wires disconnected as you wrote. But it could do so if you mishandle sense wires.
For CPX-200D i guess you would need try asking TTi.
It certainly could misbehave IF the internal connection between the sense and the output was open, i.e. the PTC blew up or was not connected for some reason.
That's why I suggested measuring the resistance from sense to Vout.
Of course, this is based on the schematics that was linked earlier in the thread, so who knows what they changed in the D model...
-
PTC blew up are was not connected for some reason.
PTC won't blow up, it will just increase the resistance until fault condition is removed.
-
The sequence of events was as follows:
-Take power connector from the table, proceed to plug in
-Before connector is mated, BOOM.jpg
-I instinctively disconnect the wire and back off, my colleague backs off too.
-I go check the supply to turn it off, realize it is already off
-Without touching anything, we take picutres of everything
-Pictures show that the channel enable LED was off, and current meter shows set current (50mA) instead of 0, which is what it would show if the channel was ON
That's why I say we're pretty sure it was off. In any case, this was destined to happen. One time we would actually forget to turn the supply off or something like that and it would have blown off anyway.
You could easily forget that you first disabled PSU before disconnecting the wire as a first measure. It's really hard to remember accurately when something like this happens. The only way to be sure would be only if you were taking video during the incident.
LOL you make me doubt now XD. The workbench we use has a table covered in antistatic mat, a zone for some instruments, a shelf for more instruments and an upper shelf where we tend to place boxes and the like. We intentionally placed the power supply in the upper shelf in a corner, to keep it as far off reach as possible to avoid accidentally touching the dials or something. I'm pretty sure I didn't have time to reach the supply before noticing it was off. My colleague says he remembers me stretching my hand to reach the supply, stopping in mid air pointing at the damn thing and saying "it's off... it's f*ing off" (not with those exact words, I curse in Spanish).
Now you'll ask how long the power supply wire is. Well, close to 1 meter. It's provided with the evaluation kit, the manufacturer says you should use it. Recommends remote sense PSU to compensate for losses in wire. We should've cut it, I know. But we decided to act as per the provided instructions.
You could easily forget that you first disabled PSU before disconnecting the wire as a first measure. It's really hard to remember accurately when something like this happens. The only way to be sure would be only if you were taking video during the incident.
That seems like quite a stretch. If something blows up just as you're plugging it in, your attention isn't going to wander back to the power supply button. You're going to yank the connector out.
This is how I deal with $1000 eval boards with goofy power requirements and no supply conditioning:
(http://www.ke5fx.com/hmc661.jpg)
The box just contains a couple of 3-terminal regulators. Nothing fancy, but they can save the chip from most mistakes made by the board designer, the PSU, or me. :-BROKE
We quite often work with GaAs MMIC's that require several power rails (some even 7 or more), both positive and negative, to be sequenced while controlling voltage and current, or else they blow. We ended up designing our own scriptable "voltage sequencing board" to avoid goofing up and blowing these devices. It contains several digitally adjustable regulators and load switches, all of them with current and voltage metering and a mircrocontroller to control the thing, and also some protection. You just load the on/off/desired working point script and press the ON button. It's been already amortized several times so far.
-
... ...
Watching - I like detective stories
Enjoy it! But it seems more like a boring comedy...
Maybe the culprit is the material of the power supply feet! Are they of plastic or rubber? We have to analyze this!
From the initial description, it seems that was a user fault. They had to test a sensitive device, so they used a power supply with remote sense. But, instead of having fixed connections for this, they decided to connect it "on fly"! The consequences are known...
We connected it on the fly to avoid any startup surprise with the supply. I thought it was more reasonable to first power-up the supply, set it up properly and then, after it's properly setup, connect the board and finally enable the output channel.
I thought it was much safer to plug in the board with the supply in a known state and with its dials locked than to press the AC switch button with the board plugged in. Moreso in this case, where there's nothing between a 1000€ IC on a 10000€ board and the supply itself. We used the remote sense not because we liked the idea (it's idiotic not to have local regulation on-board) but because it was the manufacturer's recommended way of doing things. I didn't like the idea from the very beginning, also considering that the IC can draw current in pulses.
Now that the board is ruined, if we get a replacement or if we fix this board we'll hack it off either way and install a "modchip" that I designed with proper local regulation and protection.
I don't think that is a good idea to inflate the tires of your car while driving on the highway! It is not safer than to stop in a car-station, turn off the engine, activate the parking brake ...
You have already discovered what happened, in the update of the first post:
... ...
EDIT:
Here's an update. During this time we tried simulating the board with a dummy load, consisting of an electrolytic capacitor with similar specs to those of the tantalum that blew up (electrolytics can tolerate much more without blowing up with toxic smoke) and found some interesting things.
If you proceed exactly as we proceeded when the incident happened (power supply channel turned off), nothing happens. The output stays at 0V, no transient, no nothing. No matter how you connect the supply to the dummy load, nothing happens. If the capacitor is pre-charged it just discharges with a nice exponential curve (so the output has confirmed bleeding resistors)
If you then turn on the supply with the dummy load already connected, the output just rises nicely to 1.8V, no overshoots or weird stuff.
BUT, if you turn on the power supply channel with the dummy load disconnected and you then hot-plug the dummy load, in some cases there's a several 10's of volts transient (>40V), almost 3 seconds long, and then the output stays at 1.8V. That scenario would have explained our issue... if it wasn't for the fact that the PSU supply channel was disabled. We triple-checked that it was indeed disabled before plugging anything. Both my colleague and I are 100% sure that it was off.
We'll keep investigating... (the manufacturer hasn't answered yet, we'll try new communication channels).
The matter was that when you tried to connect the power with this connector, the sense voltage dropped, due to the uncharged capacitor and the power supply reacted by increasing the output voltage, to compensate for this. Then the rest of the pins connected, giving enough voltage/power to break/burn the components of the DUT...
BTW: does the power supply deactivated its output due to over-voltage detection?
As you are continuing "playing" with this thing, try to adjust the over-voltage protection (if exists) for example to 2V and repeat the experiments. You may find something interesting...
-
I still think that no reputable PS should allow its output to go beyond a fairly small voltage delta above the setting for that channel, no matter what you hook up to the sense wires.
A quick test on this supply could check for that.
Externally apply a small negative voltage to Vsense+ relative to ground and see how it reacts.
When looking at the published schematics for the CPX200 this would lead to an unlimited output,since the external sense pin bias would completely override the internal connection between the output and the sense. (They might have fixed this in the D model)
If the OP's supply indeed behaves like this, then it probably doesn't belong in a lab.
-
I still think that no reputable PS should allow its output to go beyond a fairly small voltage delta above the setting for that channel, no matter what you hook up to the sense wires.
...
Necessary, but not sufficient. The hookup wire inductance could still be a problem.
See
http://cds.linear.com/docs/en/application-note/an88f.pdf (http://cds.linear.com/docs/en/application-note/an88f.pdf)