EEVblog #1247 - DDR Memory PCB Propagation Delay & Layout

[EEVblog]

When does PCB propagation delay matter in PCB layout?
Dave goes down the rabbit hole from DIY TTL processor design to DDR memory design and layout.
DDR memory termination.
What is a timing budget? When is it important?
How does signal integrity matter?
When do you have to do serpentine PCB traces to match trace and differential pair lengths?

Micron DDR memory timing budget design:
https://www.micron.com/-/media/client/global/documents/products/technical-note/dram/tn4611.pdf
The CIAA Project https://github.com/ciaa/Hardware/tree/master/PCB/ACC/CIAA_ACC

[BrianHG]

For me, with every DDR data signal going balanced through 1 via each, I've managed up to 250MHz DDR2 design, that's 500MTPS with a quality layout and good FPGA firmware with up to around 15mm trace differences between the shortest and longest data line. (Warning, it was a well, short trace minimal design & I was using 666Mhz ram.) Any faster, I had to match length on my traces, of use skewed IO settings on the FPGA.

Separate:
Proof measurement of the lines required matched J-Fet amplified low capacitance probes.  My signals were all well in the valid windows.  Don't waste your time trying to measure 500Mhz data transitions with normal 10x probes.  Also, make numerous un-tented GND pads around all your measurement points so you can use the J-Fet probe's tiny GND spring.  Using a GND wire & clip also is useless to verify eye timing diagrams.

[radioactive]

Nice video.

I designed and did my first DDR3 board in Eagle Cad a few years back.  I did actually use a field solver to make sure that I could account for the differences for the asymmetrical planes and a couple of vias (I'm pretty sure that this wasn't necessary as you mentioned in the video).  The board also had dual GigE,  PCIe.    To my surprise, the CPU and memory came up fine.  Somehow I managed to screw up the USB on the PCIe.  I think I ended up having to loopback something to do with the VBUS to get the CPU to enable the USB.  Anyway,  I hope I never have to do that again.  If I do,  I will be trying it in KiCad or even better, maybe I can find a place that would like to save money by buying a $30k package.  I watched a youtube video on some high-end system where the guy did a DDR3 layout in a few minutes  (obviously all kind of rules already setup), but that would be nice.

[mdijkens]

Very interesting. Never thought of these things

On a side note:
I designed a TTL computer which works fine up to 200KHz.
While that is fine with me I'm still interested to learn where that limit comes from?
Where can I start looking to find the bottleneck?
Debugging at 200KHz is not as easy as at 1 Hz 

Schematics are here: https://www.dijkens.com/mdComputer8/index.html#schematics

[asmi]

As far as impedance goes - check out this video below. It's a brilliant demonstration of waves behavior, and it's from 1959!!! 

[nctnico]

For me, with every DDR data signal going balanced through 1 via each, I've managed up to 250MHz DDR2 design, that's 500MTPS with a quality layout and good FPGA firmware with up to around 15mm trace differences between the shortest and longest data line. (Warning, it was a well, short trace minimal design & I was using 666Mhz ram.) Any faster, I had to match length on my traces, of use skewed IO settings on the FPGA.

15mm is about 95ps of delay mismatch (assuming a signal velocity of 6.3 ps/mm). On a 2ns time frame this is not significant and if you used a built-in DDR controller it will likely train/compensate the signal delays during initialisation. I recently did a DDR4-3200 layout  . I had to include the trace lengths of the BGA package in the timing calculations to get the timing right on paper. The board hasn't arrived yet though.

BTW I'm not sold on the Saturn software for differential pairs. The field solver in the CAD package I'm using produces completely different numbers.

[asmi]

15mm is about 95ps of delay mismatch (assuming a signal velocity of 6.3 ps/mm). On a 2ns time frame this is not significant and if you used a built-in DDR controller it will likely train/compensate the signal delays during initialisation.

AFAIK write and read leveling only appeared in DDR3. So nothing to train on DDR2.

BTW I'm not sold on the Saturn software for differential pairs. The field solver in the CAD package I'm using produces completely different numbers.

You're using Orcad if I remember correctly, right?

[jhpadjustable]

On a side note:
I designed a TTL computer which works fine up to 200KHz.

I had a look. You are exceeding the capabilities of the 555s. Return them to the kindergarten and replace with a crisp, clean, 50:50 square wave. I think you could much more than double the operational clock speed without touching anything else. The Nano's hardware timers are a convenient source of up to 8MHz square waves, if you swap some pins around.

To return to topic, during design of such systems, you should construct a timing budget, similar to the one Dave showed in the video. Instead of focusing on wire delays of picoseconds, you concentrate on gate delays of tens of ns. For example, at Vcc=5V±10%, the HC04 takes as much as 14ns from input stimulus to output response. The HC00 can take up to 27ns from input to output. Counters and ALUs take even longer to propagate, and also have setup and hold times relative to the clock which must be satisfied. Memories can take yet longer. All of these delays add up.

You already have a design, so instead you could compile a timing report, tracing each path through the design, starting at the rising clock edge, through the outputs triggered by that clock edge, and propagating outward and around through each other device and its consequent delays, until you've reached the next clocked input. You can mostly treat a bus as a single signal for this purpose. The path with the longest time is called the critical path, and it determines the maximum speed of the design. As a repayment for your efforts, you may find that some components and the delays that come with them are unnecessary to satisfy setup and hold requirements, and only waste time. The inverter on 8B1B's clock input, for example.

Assuming your construction is clean and neat, and the bus series resistors (edit: and LEDs) aren't dragging it down, 1MHz or better operation should be easy to achieve with just minor tweaks.

[BrianHG]

For me, with every DDR data signal going balanced through 1 via each, I've managed up to 250MHz DDR2 design, that's 500MTPS with a quality layout and good FPGA firmware with up to around 15mm trace differences between the shortest and longest data line. (Warning, it was a well, short trace minimal design & I was using 666Mhz ram.) Any faster, I had to match length on my traces, of use skewed IO settings on the FPGA.

15mm is about 95ps of delay mismatch (assuming a signal velocity of 6.3 ps/mm). On a 2ns time frame this is not significant and if you used a built-in DDR controller it will likely train/compensate the signal delays during initialisation.

It's a 10 year old design, no such thing as training/skewing adjustments.  I also ignored the DQS Latches on the reads as such a slow design didn't really need them unless I wanted to implement the 1/2 clock acceleration on the reads.  The design used an Altera Cyclone III EP3C55, over tilted to the max with a 128bit ram controller using 2x 64 bit sodim modules.  (Altera only recommended proper support on this chip for 64bit ram max on the fast bus banks)

I recently did a DDR4-3200 layout  . I had to include the trace lengths of the BGA package in the timing calculations to get the timing right on paper. The board hasn't arrived yet though.

Yup, 3200Mhz is a completely different animal compared to 500MHz.  I cant imagine the type of scope setup you would need if you ever had to verify eye diagrams.  Never mind G.Skill's successful overclocking of dual channel DDR4-4.7Ghz ram modules to 5GHz.

My design also used laptop sodim modules releasing me of the timing on the IC packages.  The modules outputs were already compensated to be relatively parallel in timing and the module output was a straight line from the sodim socket.

https://www.tomshardware.com/news/gskill-trident-z-ddr4-5000-5000mhz,36763.html

[BrianHG]

As far as impedance goes - check out this video below. It's a brilliant demonstration of waves behavior, and it's from 1959!!! 

I just love these mechanical generated illustrations of what's going on with the charge you send down electrical cables.

[mdijkens]

@jhpadjustable

Thanks for your explanation and help
When I tested max speed I used a signal generator and used a scope to check the clock and some other signals.
I thought with 200KHz there's a budget of half a clock or 2.5us which is way higher than all ns of the 74HC chips.

I am however not so sure my construction is clean and neat  and the impact of leds and (bus)resistors. Also the RC with inversed clock of the RAM chip 'worries' me..