Mysterious LED problem...could use some help

[KTP]

Me again.  I know I take more than I give on here, but maybe one day can make up for that. 

I have this very annoying problem right now in what I thought would be a simple RGB led matrix board.  The problem is that leds that are supposed to be inactive are very dimly on.  This has to be something silly stupid simple that I am just missing but it is driving me a bit crazy.  I estimate from experiments with a single RGB led on the green element that the annoying won't-stay-dark leds are only getting about 0.5 to 1 uA.  Just enough to see them in a darkish room from a foot or two away.

The 12x12 matrix is split into two 6x12 banks.  Discrete P channel, logic level mosfets drive the rows (six total mosfets, each driving one row in each bank) and the column elements are driven by six Allegro A6282 16 channel current regulated sinking drivers (using only 12 of 16 channels).   Three chips per bank, each driving either all the red, green, or blue elements.

I have experimented with the turn on, turn off times of the row driver fets, and even added a push pull driver to the rows along with a beefy 150mA gate driver all which made little difference.  The leds are being software PWM'd by an Atmel Xmega and tight code...right now hitting the rows at a fairly slow 540Hz and much faster column (for the color control).  If I slow things way way down the problem goes away (or at least can't be seen by my eye).

The row drivers are discrete Pfets, FDS9953A, 300pF gate capacitance @ Vgs -4.5V, Rds_on = 200mOhm
The column drivers are contained in the Allegro A6282 chips.

I still need to do some serious scope work, but maybe there is just something too obvious that I am missing.  I am at this moment toying with the idea that the leds are acting as capacitors and feeding others even when the pmos row driver is fully off.  Or I have some serious inductance, capacitance issues on the board layout.  0.5uA doesn't sound like much...

I have attached a simplified portion of a crude schematic to make things a bit more clear.  Thanks ahead for any help or suggestions what to try next.

[SeanB]

I would guess capacitive coupling through the reverse biased diodes, or leakage current. Try adding some pull down resistors on M1 and pull up resistors on M2 to bias the diodes harder off.

[KTP]

I would guess capacitive coupling through the reverse biased diodes, or leakage current. Try adding some pull down resistors on M1 and pull up resistors on M2 to bias the diodes harder off.

Thanks.  I have tried adding pull down resistors on M1 which actually made the problem worse in some areas (this was really strange, but maybe explainable if the PN junction capacitance of the reverse biased diodes is passing current at the frequency I am using).  The currents in question are so low it is somewhat difficult to try and pinpoint with a scope what is going on (or maybe I just need more experience).

I played around with pullups on M2 on a few pins (there are 72 total channels so doing all of them is not trivial) and it actually made the problem worse also.

[enz]

Try a Pull-Up at the Gate of M1, not M2.
I think M1 doesn't shut off fast enough.

[KTP]

Try a Pull-Up at the Gate of M1, not M2.
I think M1 doesn't shut off fast enough.

Thanks for the reply!  The board already has a 10K pullup on the gate of M1, and I have tried to increase this to as much as 500 ohms with no difference.  I currently drive the row mosfets through a 74AC138, which acts as both a halfway decent driver (+/-24mA), a level converter between the 3.3V atmel ports and the V+_LED voltage, and also reduces data lines from 6 to 3 and ensures only one row at a time will be on (in each bank).

Thinking as you did that the row transistors were not turning off fast enough and for a small amount of time was staying active after the next row's column data was loaded and latched on the A6282, I built a push pull driver capable of 150mA source/sink to drive the gate of M1.  I only did this for two rows, but even when only putting data on those two rows I still had the faint led element on problem.  You can start to see why this problem is annoying me.  .

[enz]

Did you also try a replacement for the Allegro Driver? For example a simple MosFET with current limiting resistor?
Maybe the problem is the leakage current of the Allegro Driver.

The datasheets says max. 0.5µA maybe...

[KTP]

Did you also try a replacement for the Allegro Driver? For example a simple MosFET with current limiting resistor?
Maybe the problem is the leakage current of the Allegro Driver.

The datasheets says max. 0.5µA maybe...

enz, thank you for the replies and for taking the time to read the data sheet, very nice of you.

You have come to the same idea as I, only in 10 minutes instead of a few days 

I ordered  on monday replacement driver chips (STP16CP05XTTR) which conveniently have the same pinout and serial address format as the Allegro parts, but have twice current sink capability at 100mA per channel.  The downside is that they are around $1 in quantity while the Allegro is $0.60.  I don't know if I want them to fix this or not 

I have not tried replacing the Allegro part with discrete fets and resistors...there are 72 driven channels, meaning 144 more parts and at least 720 more solder joints (on top of the thousands of solder joints I hand solder under a microscope now).  Also, I would need a shift register and latch to implement the serial functionality of the few required data lines to the Allegro parts.

Edit: with a leakage current of 1uA, it is possible the STP16CP05 parts make the problem worse, but at least this might show where the problem is.  WTB schmitt trigger LEDs 

[KTP]

Here are two videos, the first just showing two of the boards running (at 1/6 max intensity, which still saturates the camera heh..can't even see PM is yellow on camera).  The second video is trying to get a closeup of the LEDs which should never be on.  I tried to shield the other light with 1/8" plastic, but you still have to look closely at the first two leds to see the small blink of the red and green element as the ghost and pm pass by.  If a column is not on there is no led problem.  Other than this issue the prototype worked well... 5V to 15V input, RS485 communication, 6mm max thickness included pcb.

Note these videos are of two boards running side by side.  There are only 144 RGB leds per board.  Up to 256 boards can be linked together using the RS485 1/8 load drivers (and a crapload of 12V power even with the 93% efficient onboard switcher!).  I am NOT hand soldering 256 board! 

overall video:
http://youtu.be/P8oh7jHdjBc

closeup of problem:
http://youtu.be/tOfkz1V9WC4

[Rufus]

overall video:
http://youtu.be/P8oh7jHdjBc

You have said almost nothing about how you are driving the matrix.

Row and column drivers do not turn on and off instantly. When you multiplex to the next row you need to turn off all columns before for a few us and turn on the required column a few us after. If you don't you will get bleed from one column to the next (swap rows and columns if that is how you are multiplexing).

Maybe you can see this bleed, can't see much on the video. If it isn't bleeding then maybe you are doing something dumb like shifting through the register with the latch enabled.

The faster you multiplex the worse both these effects would be which seems to be the case.

[KTP]

overall video:
http://youtu.be/P8oh7jHdjBc

You have said almost nothing about how you are driving the matrix.

Row and column drivers do not turn on and off instantly. When you multiplex to the next row you need to turn off all columns before for a few us and turn on the required column a few us after. If you don't you will get bleed from one column to the next (swap rows and columns if that is how you are multiplexing).

Maybe you can see this bleed, can't see much on the video. If it isn't bleeding then maybe you are doing something dumb like shifting through the register with the latch enabled.

The faster you multiplex the worse both these effects would be which seems to be the case.

My wife is writing the software on the Atmel XMEGA, so I am with you and would much rather blame her (although she is letting me get a MSOX3024A, so maybe I will rub her feet instead  )

Seriously though, she does have delay before each row switch.  I am not sure of the present amount but we have experimented up to several usec.  It didn't seem to make a huge difference, but hard to tell with the problem being so small.  I will have her add back in more delay between row switch just to make sure the problem is not bleed.  I don't touch her code now that she switched from C to assembly (she said the AVR compiler was crap lol.  She can write 1000 lines of assembly in about 10 minutes anyway).

Edit: Also I too at first thought she was shifting through the active low latch accidentally and had her check that (plus I scoped it heh heh).  I really do think it is hardware at this point....possibly the leakage current talked about above.

[enz]

I have not tried replacing the Allegro part with discrete fets and resistors...there are 72 driven channels, meaning 144 more parts and at least 720 more solder joints (on top of the thousands of solder joints I hand solder under a microscope now).  Also, I would need a shift register and latch to implement the serial functionality of the few required data lines to the Allegro parts.

Edit: with a leakage current of 1uA, it is possible the STP16CP05 parts make the problem worse, but at least this might show where the problem is.  WTB schmitt trigger LEDs 

Of course i didn't mean to change all channels to a discrete MosFET. Just a few to try to locate the real problem.

[Crumble]

Well, this problem does sound rather annoying. I couldn't see much on the vid's, but how well is your power supply decoupling? The drivers will certainly be offset by voltage ripple. It looks like the switching MOSFETs are in the middle of the board and their gate drive might be offset by ripple in the power supply too. (Do correct me when I'm wrong, it is hard to see the layout of your board in de video).

Imho there are two things you can do quite quickly to check some possible problems:

If you have any ceramic (or similar low ESR type) capacitor in the 100nF range you can connect it to the power supply near the LED and/or appropriate driver and check if the bleeding disappears. If it does it is a power supply decoupling problem and you need to design a decoupling capacitor near where you found the most influence. This is most likely to be near the drivers (which are most sensitive to voltage fluctuations) or the MOSFETs (which are furthest away from the actual power supply and sink the most current). Sometimes an already installed decoupling capacitor is the wrong one (100pF instead of 100nF is a common mistake in prototyping).

It might also be interference the gate drive picks up through capacitive or inductive coupling of high dv/dt or high current signals. This will cause small spikes near near the edges of the transition. If so, the ghosting will disappear when you connect a capacitor in the 10nF or so range between the gate and source of the MOSFETs that filters away the spikes. It depends on the frequency and the severity of the problem how much capacitance will work before blanking out the LED altogether, this is something of an experiment. Looking at the specs the MOSFET itself gives approximately 250pF of capacitance so trying 1 - 4,7 - 10 - 47nF might work. Due to the large currents your driver is able to supply a larger value might even be necessary. This is not a solution and should be used for short turn checks to prevent excessive spike currents from flowing for too long.

These are only to check if there are spikes in the signal but are easy to check with the right (leaded) components lying around.

[KTP]

Of course i didn't mean to change all channels to a discrete MosFET. Just a few to try to locate the real problem.

Ah, ok...yes silly me.  I get all defensive when presented with a chance of messing up a board that took me 12 hours to hand solder heh heh.  Doing your suggestion would require either making up a new board, or partial new board...or trying to desolder one of the TSSOP not-very-exposed pad chips on traces that look small to a ant.

The new chips just arrived however!  With these different chips with different leakage current, perhaps it will assist in locating the real problem.  If not I may do a partial board with your suggestion.  Thanks!

[KTP]

Well, this problem does sound rather annoying. I couldn't see much on the vid's, but how well is your power supply decoupling? The drivers will certainly be offset by voltage ripple. It looks like the switching MOSFETs are in the middle of the board and their gate drive might be offset by ripple in the power supply too. (Do correct me when I'm wrong, it is hard to see the layout of your board in de video).

Imho there are two things you can do quite quickly to check some possible problems:

If you have any ceramic (or similar low ESR type) capacitor in the 100nF range you can connect it to the power supply near the LED and/or appropriate driver and check if the bleeding disappears. If it does it is a power supply decoupling problem and you need to design a decoupling capacitor near where you found the most influence. This is most likely to be near the drivers (which are most sensitive to voltage fluctuations) or the MOSFETs (which are furthest away from the actual power supply and sink the most current). Sometimes an already installed decoupling capacitor is the wrong one (100pF instead of 100nF is a common mistake in prototyping).

It might also be interference the gate drive picks up through capacitive or inductive coupling of high dv/dt or high current signals. This will cause small spikes near near the edges of the transition. If so, the ghosting will disappear when you connect a capacitor in the 10nF or so range between the gate and source of the MOSFETs that filters away the spikes. It depends on the frequency and the severity of the problem how much capacitance will work before blanking out the LED altogether, this is something of an experiment. Looking at the specs the MOSFET itself gives approximately 250pF of capacitance so trying 1 - 4,7 - 10 - 47nF might work. Due to the large currents your driver is able to supply a larger value might even be necessary. This is not a solution and should be used for short turn checks to prevent excessive spike currents from flowing for too long.

These are only to check if there are spikes in the signal but are easy to check with the right (leaded) components lying around.

Thanks Crumble.  There are 100uF ceramic caps (expensive!) next to each column driver chip connected to the V+LED rail and ground and 0.1uF caps also next to each column driver chip connected to the logic supply and ground (these chips use 3.3V logic supply and V+LED is a separate supply).  I placed the chips in the middle of the board for ease of routing, because I wanted to only use 2 layers for cheapness (maybe this is biting me in the ... now though).  I have a nice ground plane running down the middle underside, but not over the whole board in the matrix section.  The logic section is well ground planed.

Spikes on the mosfet gate....hmmm.  It could be possible...I will look into that as you suggest, thanks again.

[KTP]

To aid in the discussion, here are some closeup pictures of the front and back of the protoboard (yes, I did not clean the flux yet....but it is Kester 186 and shouldn't be a problem).  The back is a shot of an unsoldered board.  I placed the driver chips in the matrix area also because I have designed the upper logic section to easily break away and fold under the board using headers in the double row of holes in each connecting trace.  This way if one wanted to make a huge array the board spacing would work out peachy.

[DrGeoff]

A couple of scope traces would be nice to see. That way we can see the delays between the row and column switching and what the edges look like.

[KTP]

A couple of scope traces would be nice to see. That way we can see the delays between the row and column switching and what the edges look like.

When my MSOX3024A scope gets here, we will see oodles of scope traces...including exact timing of logic switch events and their analog consequences.  I am excited!  I know good equipment doesn't fix poor skills, but when I fully learn the new scope ins and outs it should be a great help on mixed signal stuff.

[DarkPrince]

Are you sure this is not a software problem? In order to prevent ghosting you need to blank the row/column being driven before advancing the column/row selection line, that way the next row/column is not being driven with the old data even for a fraction of time.

For example, my matrix follows this pseudo-code:

Load Data to Column Buffers
Enable Blanking
Jump to the next Row
Latch Data to Column Latches/Drivers
Disable Blanking

This prevents the previous line from showing up at the next line for a very small time interval. I could be completely wrong but it is possible. =)

[KTP]

Are you sure this is not a software problem? In order to prevent ghosting you need to blank the row/column being driven before advancing the column/row selection line, that way the next row/column is not being driven with the old data even for a fraction of time.

For example, my matrix follows this pseudo-code:

Load Data to Column Buffers
Enable Blanking
Jump to the next Row
Latch Data to Column Latches/Drivers
Disable Blanking

This prevents the previous line from showing up at the next line for a very small time interval. I could be completely wrong but it is possible. =)

Yes, we even just tried adding a huge delay.

Load Column data
Blank
Added 5uS delay here, made no diff.
Switch Row
Added 5uS delay here, made no diff.
Latch Column data
Added 5uS delay here, made no diff.
UnBlank

Pretty much exactly what you describe.  I think as the DrGeoff prescribed, we will need to submit the patient to the extensive probing of the MSO    Stay tuned next week, same bat time, same bat channel.

[deephaven]

Why don't you temporarily switch off the multiplexing and just illuminate one row or column and see what the adjacent LEDs are doing? This will then tell you if it's a switching problem or a driver leakage current problem.

[KTP]

Why don't you temporarily switch off the multiplexing and just illuminate one row or column and see what the adjacent LEDs are doing? This will then tell you if it's a switching problem or a driver leakage current problem.

When you step through the code the problem is not there (just illuminating one row with no row switching).

If no led in a column is ever active, that column never has this problem (just to point out that this would tend to eliminate some sort of board noise causing the problem...or maybe not...I have not had morning coffee yet).

Today I should be able to solder up the 3rd board (if i don't spend too much time on EEVblog) with the new different column driver chips...

[dcel]

Try putting a 1k or 10k pull down resistor on the drain of the p-chan fets. I had the same problem with the project I am working on and found that fixed it. Something to do with a minimum load on the p-chan that is the problem with them not turning off completely or not fast enough. They acted like there was a RC timed roll off just at the LED voltage off point. It could be just my application because I am using commercial LED stop/turn/tail lights @ 12vdc.

Might be worth a try...

Chris
 

[codeboy2k]

LEDs have an intrinsic capacitance, from about 20pf to 90pf, depending on the LED.  Including the trace series inductance, and a LED string is hard to switch off fast.

You have to turn them hard off, otherwise they tend to stay charged up, especially when you are switching them fast.

That's also why it doesn't show up when you just have one column on.  I don't think it's leakage on the board, it's more likely the PFET not turning off fast enough, or or not being fully off, coupled with the LEDs built-in capacitance and the trace inductance maintaining a current flow.   The best way to switch off the string is if you can pull it below 0 when off. This becomes more necessary as you switch at faster and faster speeds.  Done this way, you reverse bias the capacitance and it will go full off quicker.  Otherwise, work on making sure the PFET and NFET are both fully off when switching.

As this diagram shows, LEDs make nice varactors

[KTP]

LEDs have an intrinsic capacitance, from about 20pf to 90pf, depending on the LED.  Including the trace series inductance, and a LED string is hard to switch off fast.

You have to turn them hard off, otherwise they tend to stay charged up, especially when you are switching them fast.

That's also why it doesn't show up when you just have one column on.  I don't think it's leakage on the board, it's more likely the PFET not turning off fast enough, or or not being fully off, coupled with the LEDs built-in capacitance and the trace inductance maintaining a current flow.   The best way to switch off the string is if you can pull it below 0 when off. This becomes more necessary as you switch at faster and faster speeds.  Done this way, you reverse bias the capacitance and it will go full off quicker.  Otherwise, work on making sure the PFET and NFET are both fully off when switching.

As this diagram shows, LEDs make nice varactors

I did try putting a nfet along with the pfet on two of the rows in a push pull configuration thinking exactly along the lines you have.  My wife then modified the code to only switch between those two rows, and the problem was still there.  Note that for this test I also beefed up the driver to the pfet/nfet push pull such that it could drive the gate from 0.1V to 4.8V at +/-150mA.

It seemed to make no difference...there was still visible light coming from the off leds in a row when the leds in the row above that were on.

But thinking on it more, perhaps you are still correct and I actually need the low side column driver to pull hard high when off, instead of just going to a high impedance state.

Such a frustrating problem, but I appreciate your reply and it may be several things going on here.  I really need to get down and dirty with some scope time, but rather wait on the mixed signal scope and spend the meantime soldering up this other board with different low side drivers.

Thanks.

Edit:  oh, I did see that you actually said pull the line *below* zero...haven't tried that yet.

[codeboy2k]

Yeah I thought about push-pull too. I am surprised that didn't work either, and now I am curious to see what the new mixed mode scope shows you about this.

Pulling it down below zero just helps to discharge the led line harder for you.  It might work, it's worth a try, and easy to setup on a test bench with an external power supply.  The push-pull setup helps here.