High-Speed PCB layout challenges - Learning the dark art the hard way.

[hendorog]

oh wait..... I see what you are pointing out. Maybe you are correct, I may be able to simplify the caps out of the circuit altogether - getting rid of a bulky disturbance in the force.

Sent from my horrible mobile....

Yeah - and I didn't realise it when I posted but you can connect VCC_O to the 1.8v as it is just for the outputs.

[rx8pilot]

That is a solid catch, thank you.

Sent from my horrible mobile....

[rx8pilot]

I doubt Carlos can afford to hire Lee Richey since he (Carlos) is a one-man shop.

I will confirm that doubt and present it as absolute fact, lol. Books, app notes, You Tube, intellectually generous people are forums, etc are my primary strategy.

Also note that proper POWER INTEGRITY Is required for HS design since HS ICs need proper PI design to work reliably. This subject is more complex since it needs advanced software such as Hyperlynx to simulate PI of the various power/ground plane layers. I have done such PI simulations for a few of my customers (on consulting basis) and it becomes more common that modern large ICs souch as SoC's and FPGAs don't work reliably due to incorrectly (under) designed power planes and decoupling capacitor placement. The speedingedge books explain this in detail but so do various app notes from Altera and other IC vendors. Bogatins and Peng Li's books also go into this subject ad nauseum...

Absolutely. I have spent the past couple of years doing DC power electronics and power management. I have not, however, developed any PDN directly for high-speed. My projects are all for primary power that feed a local PDN system. It is good and relevant experience to have - but I still have things to learn about powering this type of circuit.

[Someone]

How, and where, are you connecting your probe? Could this just be a probing artefact rather than an actual SI problem when the scope isn't there?

I only have one and wishing I had a second to double probe the circuit and see if the probe is doing much damage as a sanity check.

Simulation says the probing should have a small but measurable impact on the signal, which should be visible clearly in the frequency domain and cause quite different looking eye diagrams at each probing point. Did you have a consistent test signal?

There's a simple standard technique to see how much a probe is affecting a signal. You add a second probe alongside the first probe, and see the change in the signal measured with the first probe.

I'm well aware of this rediscovered technique (care of Keysights recent marketing) but the OP specifically doesnt have a second probe and said the signal looked the same at each location which it theoretically shouldn't. The important information is what the SI looks like at the receiver and the probing will be changing the measured result so its useful to have some other ways to assess its impact rather than buying another probe as a dummy. If the eye looked the same at all the probing locations then something is seriously wrong.

[Pitrsek]

Two years ago the price of Lee Ritchey 3day seminar was cca 2200eur. Dunno what is price now. But it was packed with information. I don't know how much HS design you have in front of you, but it might be worth it. Take a look at his website, lot of good info there. Books are nice as well.

Regarding the PDN, look at videos/articles from Steve Sandler, also literature from Istvan Novak is very nice. Last year Steve did a one day PDN bootcamp - price was 250eur. And it was really nice. Great guy who knows his stuff. I think he collaborates with Keysight - so you can ask your local contact if there is some training planned in your area.

The thing is that for PDN(well, actually for everything you do...) you need to be able measure what you are doping. In this case it means SA with TG.
Or better "low freqency" VNA - this one is more versatile, cheapest one I know is BODE 100. Actually having both will give you some serious measurement potential....

[rx8pilot]

Regarding the PDN, look at videos/articles from Steve Sandler, also literature from Istvan Novak is very nice. Last year Steve did a one day PDN bootcamp - price was 250eur. And it was really nice. Great guy who knows his stuff. I think he collaborates with Keysight - so you can ask your local contact if there is some training planned in your area.

The thing is that for PDN(well, actually for everything you do...) you need to be able measure what you are doping. In this case it means SA with TG.
Or better "low freqency" VNA - this one is more versatile, cheapest one I know is BODE 100. Actually having both will give you some serious measurement potential....

I met Mr. Sandler about a year ago. He was (no surprise) a wealth of practical information. Needless to say, I have his book which is a nice bridge from theory to practical reality in power supply design. I looked closely at the Bode 100 and nearly got one on eBay a couple of months ago - it disappeared before I could move on it. They rarely come up on the used market, at least at a good price. I have such an enormously long list of very important equipment needs - it is a challenge to manage that as funds allow. Right now, a Markforged 3D printer may have to be the next thing which will push EE gear down the road along with everything else.



This book should help......arrived a few minutes ago.

[rx8pilot]

Quick update -

The book mentioned above is excellent so far. Very practical and easy to follow, but appears to have enough detail to be truly useful.

I met with an engineer from Samtec, the company that makes the BNC connectors and gathered a wealth of information. He had a 20 Ghz Keysight PNA series VNA with a very rare 12Ghz 50 Ohm to 75 Ohm matching pad and cal kit. We were able to look at how to best launch a 12Ghz signal from the connector to the PCB. We looked at good and bad on the VNA. We talked about various PCB materials, RF simulation, etc. The real good part was that he and others at Samtec can take my project and model it as well as practically test it with their fully configured lab. the equipment and talent are miles out of reach for me so this is an amazing resource. He set up a demo where a 12Ghz signal was run through about 14 connections, cables, and PCBs - the eye was still open at the end and able to be decoded without errors. That was pretty impressive. I was expecting the signal to be mush by the time it went through that many transitions.

I also met with engineers from Semtech, Texas Instruments, and Macom who were also a wealth of information on how to deal with the various challenges in high-speed digital. Semtech was the most generous about offering practical assistance and reference designs. TI was pleasant, but did not have or did not offer as much assistance. For example, Semtech can help review designs as well as validate the designs after they are built in a purpose built lab. TI offered design help, but did not offer any validation service. Macom (makes similar silicon) was the least friendly from a design perspective, but that may have been the person I was speaking with. It felt like he had not slept in weeks and was drinking since 9am.

Anyway - the conversations I had with the collection of engineers this week and the various design assistance available leaves me encouraged that I will be able to get high-speed IO sorted out for my FPGA projects and other things. As difficult as the learning curve has been so far, it really is a lot of fun. Expecting to submit the next spin of the PCB tomorrow, fingers crossed I will at least see some improvements. If I can operate at up to 3Ghz, it will be marketable. If it is marketable, I can better pay for the various test gear and learning curve needed for the road ahead. I will try to post the latest PCB layout before I submit it.

[KE5FX]

I met with an engineer from Samtec, the company that makes the BNC connectors and gathered a wealth of information. He had a 20 Ghz Keysight PNA series VNA with a very rare 12Ghz 50 Ohm to 75 Ohm matching pad and cal kit. We were able to look at how to best launch a 12Ghz signal from the connector to the PCB. We looked at good and bad on the VNA. We talked about various PCB materials, RF simulation, etc. The real good part was that he and others at Samtec can take my project and model it as well as practically test it with their fully configured lab. the equipment and talent are miles out of reach for me so this is an amazing resource. He set up a demo where a 12Ghz signal was run through about 14 connections, cables, and PCBs - the eye was still open at the end and able to be decoded without errors. That was pretty impressive. I was expecting the signal to be mush by the time it went through that many transitions.

This would've made a killer video.  It would certainly have made Samtec look good.

[rx8pilot]

No doubt, I wish I had my camera with me. The engineer was the one that designed the specific connectors I am using so there was no guessing. I think the VNA was $150k with all the accessories, making it the  most expensive piece of test equipment I have ever touched. To be clear, I was lost on how to use it, but I did touch it just to say I did,  lol.

Sent from my horrible mobile....

[rx8pilot]

The latest edits:

Changed the BNC footprints to a better geometry. The dimensions are not perfect, but the general shape is in place. Hopefully, Samtec can provide some help this week. I did clean up the passive termination components quite a bit to get a better signal into the equalizer.

GS6042 Datasheet says:

"The differential output can be DC-coupled to Semtech’s
reclockers and cable drivers, as well as industry-standard
CML logic by changing the voltage applied to the
VCC_O pin. In general, DC-coupling to any termination
voltage between +1.2V and +3.3V is supported."

I changed the power distribution network to make VCC_0 1.8v and DC coupled the differential output to the reclocker GS6151 - obviously a lot less to go wrong with no impedance disturbing caps in the way. Thanks for pointing that out. I also read a bit more and figured out that I can DC couple from the reclocker GS6151 to the cable driver too, cleaning that up. The top ground fill is fenced with vias along the high-speed data and the distance between the microstrip and ground is about double the trace width.

One of the biggest problems was that the stack-up on the PCB was not what I planned for. The actual impedance was off substantially. I pinned those down for the next round without going to a controlled impedance PCB process.

There were a lot of other little fixes too, but probably not signal integrity related. I had to move to a switching supply to get the 3.3v because I needed a much wider range of input voltage. The original design was just LDO linear regulators and I was powering from 5v max. Now I can go up to 30v or so without smoke. I have some concern about the EMI of the new switcher, it is possible I will need some additional filtering. At this point, all I really have are input/output caps and ferrites on both ends. I have all the tools to measure power noise and response at least, so I can test it in a meaningful way.

Hope this is going in the right direction.

[AndyC_772]

What are the components in series with your signals, right next to each of the three BNC connectors?

I'm not saying it necessarily is in your case, but one trap which people do sometimes fall into, is treating a connector as though it's a driver or receiver, and placing termination components accordingly. But they're not, of course; the actual driver and receiver are just at the other end of the cable, and if the cable and connector have the same characteristic impedance as the trace, then from a signal integrity point of view, they're just an extension of the transmission line.

[Rerouter]

I'm seeing quite a few stubs in your top ground fill, most of them you should be fine to crop out and the ones that go to a component can be necked.down to the pads. Equally any pad that goes to ground should have atleast 1 via right next to it ideally towards the direction of the signal on the other side, or opposed, to keep your loop area as small as possible,

At your speeds, return paths matter, even the low speed ones as a loop can be a recieving antenna as much as a transmitting

[rx8pilot]

What are the components in series with your signals, right next to each of the three BNC connectors?

I'm not saying it necessarily is in your case, but one trap which people do sometimes fall into, is treating a connector as though it's a driver or receiver, and placing termination components accordingly. But they're not, of course; the actual driver and receiver are just at the other end of the cable, and if the cable and connector have the same characteristic impedance as the trace, then from a signal integrity point of view, they're just an extension of the transmission line.

Those parts are the return-loss compensation and the termination. The top-left BNC area is the input and the bottom-left and bottom-right BNC's are the outputs. I have already made some changes to the layout since the last image, but the parts have to stay. In theory, the BNC's would be impedance invisible but the right angle through-hole models not so much. I am also stuck with routing the signal on the wrong side of the PCB which creates a stub as the center BNC pin goes through the board. The mechanical restrictions are getting in the way of signal integrity to some extent. The requirement for this first pass is only up to 3Ghz with fairly short cables - offering some forgiveness. I can work on 6Ghz and 12Ghz on a later revision where it gets more and more sensitive.

I'm seeing quite a few stubs in your top ground fill, most of them you should be fine to crop out and the ones that go to a component can be necked.down to the pads. Equally any pad that goes to ground should have atleast 1 via right next to it ideally towards the direction of the signal on the other side, or opposed, to keep your loop area as small as possible,

At your speeds, return paths matter, even the low speed ones as a loop can be a recieving antenna as much as a transmitting

True - I went through and made some adjustments to the top ground fill to reduce stubs and added more stitching where it seemed to be needed. I blended the high-speed traces into the passive components and made sure there are GND vias next to each high-speed i/o pin. Per the suggestions of Semtech and a couple of other app notes - I used cutouts on the high-speed parts. Considering that the original layout actually worked, but just with some ringing - my guess is that the long list of improvements should make it pretty decent - at least up to 3 Ghz. To make the higher frequencies work, I will need to get the BNC launch dialed in a bit better. I recently found some parts that have the return-loss compensation built in and the the re-clocker / cable driver have been combined. That eliminates a lot of variables and drastically tightens up the layout.

[rx8pilot]

Update with REV 2 PCB:

Summary: Not much changed even after making a substantial number of changes to the PCB layout.

Change highlight include removing AC coupling caps on high-speed differential lines, EQ chip GS6042 power was re-routed to drive the output at lower voltage, differential trace geometry updated to match the PCB stack up, added guard vias, lots of power related tweaks, BNC footprint fixes, 75 ohm single ended path layout fixes, and more.

I initially went through to look at each power pin since the linear 3.3v supply was swapped for a switcher - Linear LT8607. The 3.3 is rather clean down to a few millivolts, filtered with SMD ferrites, and bypassed very close to all power pins. Seems to be working well enough for the task.

The two high-speed traces are behaving almost identical to the previous revision - ~5Ghz ringing on over/undershoot. The signal is still readable at the end, but still quite a mess. I briefly had a commercially sold PCB with similar hardware on the bench. I was able to probe that to see if my probe was potentially causing the distortion - it was was far cleaner as expected. The single ended input looks good right at the termination point of the EQ chip - amplitude, rise time, and no ringing there. The output of the EQ GS6042 and RECLOCKER GS6151 look very similar and appear to have a rather severe impedance mismatch. Both of these chips are driving the signal into an internally terminated receiver. That only leaves the PCB traces to have a large impedance deviation (I think). I hope the via placements are good enough for signal returns.


I was not expecting this iteration to be perfect, but am definitely surprised how little impact the layout changes made. It seems like a barrel if issues were addressed and at a minimum - it should behave differently if not better. Spending a lot of time learning how to determine the inductance of a via, but the issue may be much more fundamental than that. Something big is sitting right in front of my face?

[T3sl4co1l]

Evidently you changed everything BUT the one thing that has an electrical length of 5GHz.

Tim

[rx8pilot]

Evidently you changed everything BUT the one thing that has an electrical length of 5GHz.

Tim

Yep - I am stumped and back into the data sheets, books, and app notes looking for the smoking gun. At the moment, I am setting up a SPI interface with a Bus Pirate to be able to change some of the GS6151 internal parameters. One necessary change is setting the input for DC coupling although I doubt that is part of the ringing problem. I will also have access to output swing, de-emphasis, pre-emphasis, internal eye monitor.

If the issue is simply an impedance mismatch on the PCB traces, I am wondering how far off it wold have to be to ring the way it is.

EDIT:
I don't think the Bus Pirate can cope with 1.8v SPI 

If anyone has a quick suggestion to send/receive some basic commands and data over 1v8 SPI. I don't have anything to build a level shifter - although I am digging through my spare silicon right now.

[hendorog]

You could remove the components in the signal path one by one and terminate with resistors. Combined with your probing you should be able to isolate the issue.
i.e. 75 ohm resistor instead of the BNC output and then 100 ohm resistor to terminate the diff pairs as you move back down the path.

Good info that the probing is not causing the problem as that gives you confidence that there really is an issue somewhere.

[rx8pilot]

This is a shot of the commercial reference PCB I had the other day. This is a middle of the road product overall. The signal integrity is much better than what I have on mine so far - but still not a model of perfection. The better the signal, the longer the cables can be and fewer repeaters will be needed. These systems are also regularly used with poor and abused cables among other challenges so have as much headroom as practically possible is important.

EDIT: Added image
I overlayed the commercial reference with my circuit - the ringing on the commercial circuit is much lower in amplitude but similar in period.

[KE5FX]

I haven't seen your schematic, but one quick suggestion is to look for places where you can/should be using source termination, but aren't.   That kind of ringing happens when a slightly-mismatched signal path or transmission line (read: all of them) is driven harder than it needs to be.  Series R at the driver end can help by improving the termination seen by the initial reflection.

On the other hand, if this is an LVDS or other differential connection where source termination isn't appropriate, you may not have a problem at all, because the margins are better than they look.

[rx8pilot]

I haven't seen your schematic, but one quick suggestion is to look for places where you can/should be using source termination, but aren't.   That kind of ringing happens when a slightly-mismatched signal path or transmission line (read: all of them) is driven harder than it needs to be.  Series R at the driver end can help by improving the termination seen by the initial reflection.

On the other hand, if this is an LVDS or other differential connection where source termination isn't appropriate, you may not have a problem at all, because the margins are better than they look.

There is a schematic in the first post, first image. The only significant change is that the AC coupling caps have been removed. In theory - the EQ Chip GS6042 has internal source termination as well as the next chip GS6151. The whole family are CML logic on the high speed lines and the data sheet (as well as manufacturer reference designs) are connected directly to one another with no external slew rate or current control.

I used a couple of differential microstrip calculators online and could have gone wrong there.

1oz top copper - 6.9mil dielectric - 1oz GND PLANE copper - 44 mils dielectric - 1oz PWR copper - 6.9mil dielectric - GND/low speed signals
dielectric constant 4.29
differential trace 8mil spacing 7 mils

Hoping that is close to 100ohm impedance

[BrianHG]

Your circuit appears to have a faster rise and fall than the commercial reference.  When it comes to ring oscillation alone due to circuit design, I would expect to see the same speed, or, slower.  With this faster rise and fall, which may be even faster hidden due to the influenced ringing, this may be happening within you probing setup or lowering the bandwidth of your HW or pcb would soften the blow to match the commercial reference matching it's ringing.  This can happen alone just due to the thickness of you signal traces and the characteristics of the PCB itself.  Can we see a picture of your probing setup & maybe a photo of the reference PCB.

[Rerouter]

At a glance I find it very odd that the ringing frequency looks very close on both circuits, i dont suppose you can measure that frequency and see if your test setup has lengths approaching those wavelegths?

[rx8pilot]

Your circuit appears to have a faster rise and fall than the commercial reference.  When it comes to ring oscillation alone due to circuit design, I would expect to see the same speed, or, slower.  With this faster rise and fall, which may be even faster hidden due to the influenced ringing, this may be happening within you probing setup or lowering the bandwidth of your HW or pcb would soften the blow to match the commercial reference matching it's ringing.  This can happen alone just due to the thickness of you signal traces and the characteristics of the PCB itself.  Can we see a picture of your probing setup & maybe a photo of the reference PCB.

Just to be totally transparent - the overlay was stretched vertically to match the amplitude a little better. The goal was to show the similarity in the ring period, but that may have made the rise time stretch a little as well. The rise time may is likely much closer than the altered image depicts. I thought I was going to have that device for a couple of weeks, but it had to go. I think I can get another one to compare in a more scientific and planned out manner. The one shot I saved was just a quick 'jam the probe in there and see what we see' kinda thing. The reference PCB was designed for 3Gbps and the chips I am using are capable of 6Gbps. I an only testing them with a 1.485Gbps test signal.

At a glance I find it very odd that the ringing frequency looks very close on both circuits, i dont suppose you can measure that frequency and see if your test setup has lengths approaching those wavelegths?

The ring frequency is right around 5Ghz. I am manually probing with a Keysight N2752A with the browser tips as shown in the attached image. I am probing the pins directly on the QFN packages under a microscope - quite a challenge to hand probe .4mm QFN pins that are side by side. Adding any test points would never work so I guess this is how the big boys do it. The probe has solder-in wire tips but they roll off at 3Ghz so I would be attenuating the ring if I did that.

....aaaaand yes, I did not expect to see the same thing. Another detail is that the ringing seen on both high-speed traces is nearly identical. The GS6151 chip in the middle of the chain re-clocks and recreates the signal before sending it through a longer trace into a different device. The differential microstrip structure is different in that area of course. Is it possible for the probe to damage the signal this bad? I REALLY wish I had a second high-speed probe.

[BrianHG]

The reference PCB was designed for 3Gbps and the chips I am using are capable of 6Gbps. I an only testing them with a 1.485Gbps test signal.

Even with your resizing, the double speed chip does clearly depict a faster rise and fall time as the source is deliberately brought to as fast as possible rise and fall time even if the source is only 1.485Gbps.

What's the bandwidth & load capacitance of your active probe?
The ring is beginning to look like the inductance of your trace + capacitance of the probe as it is ringing in the 5GHz region driven from an IC capable of 6Gbps.
It may be entirely possible that the ring you see may not be there on your PCB.
Do you have access to a higher frequency probe?
A few years back, I had a similar issue, though 750 Mhz and 1 Ghz, with my Tektronix active J-Fet probes, I had 1 which was 1.5Ghz and 3 at 1Ghz, both rated at less then 0.5pf.  The 1.5Ghz had less bounce testing my DDR3 buss data lines on faster memory chips and apparently less capacitance at the tip.

[rx8pilot]

The probe is officially rated as 6Ghz but apparently does not hit 3db until 7Ghz. 200k differential and only 700fF capacitance.

I am going to check and see how much the browser tip may add. Still, it could be a measurement error or at least exacerbated by the measurement.

EDIT:
The 700fF capacitance is with the browser tip I am using - that is rather low.

Even with your resizing, the double speed chip does clearly depict a faster rise and fall time as the source is deliberately brought to as fast as possible rise and fall time even if the source is only 1.485Gbps.

I am not positive, but I think the chips match the rise time to the supported data rates. As the data rate increases, the rise time does as well. Checking to confirm that.