EEVblog Electronics Community Forum

Products => Test Equipment => Topic started by: jjoonathan on October 10, 2020, 05:12:22 am

Title: PS350 5000V Power Supply
Post by: jjoonathan on October 10, 2020, 05:12:22 am
I recently bought a Stanford Research Systems PS350 high voltage supply in "unknown" condition and fixed it. The problem was a blown JFET buffer amp on the divided-down voltage and current monitors from the high-voltage section (see attached). After repairing the supply I set about putting it through its paces -- holding voltages from -5kV to 5kV while probing with a HVP40, pushing milliamps through mega-ohm resistors, some hipot testing to the point of arc-over... and on the second arc it killed itself. The display froze with gibberish on it and a power-cycle didn't even bring back the gibberish, it just sat there, display off, like a brick.

EDIT: clarification, not like a brick -- it made the telltale "HV charging" noise, despite nothing on the display, so I removed the orange wire to disconnect the HV section, and *then* it sat there like a brick.

My first question is about expectations. I went in with the expectation that a lab power supply was supposed to handle short-circuit conditions without killing itself. Does that not apply to high voltage power supplies? Am I correct to conclude that the supply likely has an additional defect that made it especially vulnerable (perhaps some bad decoupling around the HV interface)? Or is this just a limitation that's standard on this class of devices and I should learn it and live with it?

My second question is about debugging the new failure state. With my thermal camera I caught a quad-comparator shorting the 5V rail, so I replaced it, and now all power rails are good. Unfortunately, the same "completely bricked" symptoms persist. It looks like the Z80 processor is just spinning endlessly through its address space because something is dragging the data bus low. On the scope you can see the EEPROM valiantly trying to pull the data lines high at certain addresses but it never gets more than 100-200mV off ground unless you lift an EEPROM pin, at which point the floating pin has no trouble producing 5V waveforms. Right now my plan is to painstakingly trace everywhere the data bus goes (I don't have schematics) and start cutting traces until I figure out what's shorting the bus. However, I expect this approach to be a right PITA and this seems like a problem that would have been common in the era of 74 series logic and parallel busses -- so does anyone have special strategies they'd like to share for attacking "who's sitting on the bus" challenges?
Title: Re: PS350 5000V Power Supply
Post by: picburner on October 10, 2020, 06:35:02 am
A little documentation on the SRS PS300 power supplies family (including schematics) can be found here (http://www.electronicsandbooks.com/eab3/manual/index.php?dir=Hardware%2FS%2FStanford+Research+www.thinksrs.com%2FProduct%2FPS300+PS310+PS325+PS350+Power+Supply%2F)
In case you have fried the eprom I am attaching the dump of it.
Title: Re: PS350 5000V Power Supply
Post by: coromonadalix on October 10, 2020, 11:37:04 am

My first question is about expectations. I went in with the expectation that a lab power supply was supposed to handle short-circuit conditions without killing itself. Does that not apply to high voltage power supplies? Am I correct to conclude that the supply likely has an additional defect that made it especially vulnerable (perhaps some bad decoupling around the HV interface)? Or is this just a limitation that's standard on this class of devices and I should learn it and live with it?


It may not work the same way as the standard low voltages psu's,    it depend how they make / get the high voltage output ( buck boost, voltages double / tripler ...),   you're playing with lots of voltage, even with low current at the output,  the limitter could be unable to react fast enough ?   and thats a lot of voltage range(s) to cover, and it may react weird with say inductives or capacitives loads ??   
Title: Re: PS350 5000V Power Supply
Post by: coromonadalix on October 10, 2020, 12:05:47 pm
For your debugging sessions, i would not start cutting traces  and creating more damages,   list all the part numbers and check the datasheets, you can spot  bus buffers   and other  memory (eeprom, rom ...)   

Around a cpu  normally you have some bus buffers, bus expanders, octal flip flops like 74ls374, when you spot the cpu model / brand, you can google it  and see application notes  etc ... it would help figure it out in a way.

If you are equippend with a desoldering pump, i would carefully remove some chips and put them on ic sockets

You have a Zilog cpu with an eeprom (28 pins) and a sram i think (24 pins) next to the eeprom. I some very old stuff at my job, the sram where getting bad over time.
Title: Re: PS350 5000V Power Supply
Post by: coromonadalix on October 10, 2020, 12:13:23 pm
You have a thread here : https://www.eevblog.com/forum/repair/quick-repair-standford-research-ps350-5kv-5ma-(fixed)/ (https://www.eevblog.com/forum/repair/quick-repair-standford-research-ps350-5kv-5ma-(fixed)/)

Niice   lots of pics

It should be relatively easy to follow data paths,  we can get an idea  thru  back lighting the pcb,  take you time, draw some schematics on paper like you did, with the datasheets  you'll eventually get a good idea  if the data bus is pulled down, a short ?,  a supply voltage is missing etc...
Title: Re: PS350 5000V Power Supply
Post by: TurboTom on October 10, 2020, 12:21:43 pm
That 10nF capacitor across the HV branch of the sense voltage divider would make me a little nervous... Even though there is a pair of back-to-back zeners to limit the voltage across the Vsense lines, I'm not sure if they will be fast enough if there's an arc across the output terminals. The Vsense buffers (U109, and I assume that's the one that you found defective) are coupled to these lines just via a pair of 680 Ohms and 22n in parallel, IMO not enough to prevent it from damage in such a situation.

With the schematic available, it shouldn't be too difficult to check all the chips that may drive the data bus for a fault and maybe also the address decoders. I'm sure that you'll find the culprit...  ;)

Title: Re: PS350 5000V Power Supply
Post by: coromonadalix on October 10, 2020, 12:32:53 pm
Ps300  serie no schematics attached,  it may  help a little

EDIT just found this one ... attached --- many models schematics
Title: Re: PS350 5000V Power Supply
Post by: TurboTom on October 10, 2020, 12:43:51 pm
Yes, @picburner already published a link to these files in the first reply...  ;)  But this way, it's directly accessible.
Title: Re: PS350 5000V Power Supply
Post by: coromonadalix on October 10, 2020, 01:00:19 pm
Oh  my bad  sorry  loll  i knee myself  :palm:   |O      have to refresh my page more often loll

Thks  @picburner

I did saw a few pages at EEVblog    of   S&R  psu's  and some hacks   rom dumps  etc...  they seems  good beasts ?


Even saw Dexter has made a software to help calibrating some psu ??
https://www.eevblog.com/forum/repair/quick-repair-standford-research-ps350-5kv-5ma-(fixed)/25/ (https://www.eevblog.com/forum/repair/quick-repair-standford-research-ps350-5kv-5ma-(fixed)/25/)

A repair video by "the signal path" ...
https://www.youtube.com/watch?v=09otx2Pfphw (https://www.youtube.com/watch?v=09otx2Pfphw)
Title: Re: PS350 5000V Power Supply
Post by: jjoonathan on October 11, 2020, 03:24:22 am
Thanks for the schematics! Those really make a difference.

Quote
the limitter could be unable to react fast enough

Well, with that big capacitor on the output I'd consider the current limit more of a time-averaged thing than an instantaneous thing. Disappointing, but not unexpected. All of my other power supplies behave the same way, except for my SMU, but it only goes to 200V and I pay dearly for the privileges it affords me.

Quote
It should be relatively easy to follow data paths,  we can get an idea  thru  back lighting the pcb

I decided to go begging for schematics after losing the trace under a chip. Is there a trick to work around this? Other than lifting the chip... or buying an xray machine?  ;D

I could really use an xray machine. Maybe someday.

Quote
If you are equippend with a desoldering pump, i would carefully remove some chips

Unfortunately I don't. Fortunately, it looks like you called it -- everything else on the data bus, aside from the socketed EEPROM, socketed SRAM, and socketed GPIB controller, is just a 74 series buffer or tristate. After cycling through various removal strategies (I used BiIn alloy for the JFET buffer, wick and wiggles on the comparator, and a snap-type solder sucker on the first tristate) I decided to just shotgun the rest of them (pics attached), adding sockets where possible. I won't learn which one failed, but if I had to guess I'd pick the one connected to the +/- detect from the HV compartment. In any case, with the buffers and tristates removed, the Z80 isn't sweeping aimlessly through its address space anymore and the EEPROM is able to pull the data lines up to a full 5V. Of course, with all the I/O buffers removed it's still catatonic, but that can't be helped until the chips arrive. I'll update you all then :)

In the meantime, strategy. I want to use this thing to test arc-over. Regardless of whether it killed itself because of a design flaw or because one of its clamps failed, I'm interested in fixing it until it has such capability. My plan is to set it to a low voltage and use an IXBH12N300 I have sitting around (plus TBD gate driver, plus signal generator) to hit it with a train of load pulses and see if I can catch inappropriate spikes happening at the outputs of the HV module. I suspect TurboTom is right to be suspicious of the bypass capacitor on the V_SENSE divider, especially given the state in which I received the instrument. In any case, I'll let you know how it goes!
Title: Re: PS350 5000V Power Supply
Post by: coromonadalix on October 11, 2020, 05:11:02 am
I would try when you replace all the chips  the sram chip, try to start the psu with this chip removed ?

As i wrote earlier, i did had some problems with theses,   in the machines we build at my shop, when we do a factory reset,  if they go bad,  it take a very long time to reset and sometimes it fail.


In the video at the end :  we do see a good arcing and the psu survive this test ...
Title: Re: PS350 5000V Power Supply
Post by: jjoonathan on October 13, 2020, 11:36:52 pm
It lives! The shorted flipflops/tristates must have been the last victims of the surge. The SRAM seems fine, but it's useful to know that you've seen them die before. I'll squirrel that knowledge away.

Thanks to smgvbest and Dexter2 in the other thread I now have GPIB working as well! The original calibration was pretty much spot on, at least after the JFET replacement buffer arrived. I was previously using a LM358 as a temporary stand-in and the calibration was very much *not* spot-on with that substitution, but now it's good to a least significant digit up to 1kV and 5mA so I just copied the entire calibration block over to the new firmware.

Now I'm just waiting for a gate driver to arrive so that I can build the pulse load and start tracking down how these transients leak from the HV section into the main section. I'll post an update when I've figured it out.
Title: Re: PS350 5000V Power Supply
Post by: coromonadalix on October 14, 2020, 01:32:22 am
 :-+
Title: Re: PS350 5000V Power Supply
Post by: jjoonathan on October 18, 2020, 02:33:39 am
It looks like there are significant common-mode transients on the current-monitor and voltage-monitor coax cables, moreso on the voltage monitor. 20V transients when the output is charged up to 500V. That's only a tenth of the maximum output voltage yet the transient already exceeds the +-15V rails on the buffer amplifier. Not good. The cables are put through a toroid, but only with 2 turns, and the current and voltage monitor cables share a toroid, so I wonder if it just isn't enough. I've ordered some new toroids to see if I can do better, but I'm not a toroid expert so I just picked one that mentioned roughly correct frequencies and plan to make more than 2 wraps. I also plan to separate the current and voltage monitor cables to prevent any transforming nonsense. My worry is that the voltage divider and the buffer that monitors it are already high impedance, so I'm not sure the high common-mode impedance from the toroid will actually attenuate anything unless paired with extra capacitance or clamping. We'll see.

In addition to the common-mode transients that happen during discharge (~1us), it looks like the voltage divider bypass capacitor spotted earlier by TurboTom is also an issue (~2ms). The transients are 5V for 500V of output, which again is 1/10 of the rated max. The zener clamps should kick in at ~8.5V (edit: actually ~15V). Therefore, I'm also thinking about clamping to the +/-15V rails (and probably shoring up the bypassing for good measure) even though it will probably require recalibration.

These transients go pretty far out of spec for the opamp. I wonder if most PS350s have better clamps/toroids that prevent the transients or if all PS350s have these transients and they usually aren't a problem because of unspecified variation in the opamp ESD protection. In any case, it doesn't seem like a terribly robust design.
Title: Re: PS350 5000V Power Supply
Post by: jjoonathan on November 02, 2020, 04:45:13 am
I added (independent) ferrites and clamps. In particular:

I_SENSE Shield: Shorted to ground in front of resistor
I_SENSE Center: Clamped to ground behind resistor with anti-parallel-diode-pair (±250mV signal, ±600mV clamp)

V_SENSE Shield: Clamped to ground in front of resistor with anti-parallel-diode-pair (±250mV signal, ±600mV clamp -- yes, this is I_SENSE Center in disguise)
V_SENSE Center: Clamped to ground behind resistor with back-to-back zener (±8.5V signal, ±10V clamp)

These additions sufficed to eliminate the short circuit fragility that plagued the device. I have charged it to 5kV and shorted it >20 times and it has not killed itself, shown display corruption, or misbehaved in any way. Compare to before, where the first 5kV short caused display corruption and the second killed it. Oscilloscope traces verify that the I_SENSE and V_SENSE clamps are holding (shown) and that transients do not escape on any other wires leaving the HV section (not shown).

It looks like a fix!