DDR3 initialization sequence issue

[BrianHG]

tIS is measured with reference to your DDR_CK output.
Where is your DDR_CK signal in that simulation?

If you are doing a true Gate-Level timing simulation, the output and input timing will not match infinite and perfect IO speed of your old RTL simulations within Modelsim depending on the way you programed your IOs.

Good luck.

Note that in my sims, I was using Altera's authentic DDR-IO buffers & PLL to generate the outputs to feed Micron's DDR3 model.  I was not using a home-made simple logic DDR buffer or software generated PLL.

[promach]

In this case, how would I go around the IO timing issue resulted from the use of vendor's DDR-IO buffers and PLL primitives ?

Note: I am using clk_serdes from PLL to drive CKE , RAS_N , and CAS_N signals

[BrianHG]

In Quartus, the test bench simulations are a separate entity from a complete FPGA project.
I would normally design my FPGA and do a full compile then run a gate-level timing simulation.

My top_tb.v is a separate entity which only has access to the IO pins of the FPGA project.

The actual IO pins which are wired on my PCB to the DDR3 ram chip are what are wired to Micron's DDR3 model.  This is the only way you can see the true timing at the IO pins in the DDR3 controller.

In my RTL sims, I am doing the same thing for just the DDR3 IOs, however, gate-level timing sims does alter the timing of the read data.  Though, my power-up auto-read-phase pll-tuning function is wide enough to catch and correct for this automatically during initialization.

This method offers good true timing sims with ModelSim with the older FPGA like Cyclone IV and earlier.  This method is now deprecated with the latest generation of FPGAs from Intel for strict board-timing analysis tools which no longer support this type of simulation.

I do not know how this is done in Xilinx.

[BrianHG]

Remember, for true timing sims, you need to compile an actual FPGA as that timing is affected by things like the FPGA grade, IO voltage standard, and which pins you are actually using and where they are located on the FPGA.

This true timing information just isn't available to Modelsim under a normal logic RTL compile.

[promach]

however, gate-level timing sims does alter the timing of the read data.  Though, my power-up auto-read-phase pll-tuning function is wide enough to catch and correct for this automatically during initialization.

What do you exactly mean by power-up auto-read-phase pll-tuning function ?
Is it some MPR read functions ?
Or some other things ?

[BrianHG]

however, gate-level timing sims does alter the timing of the read data.  Though, my power-up auto-read-phase pll-tuning function is wide enough to catch and correct for this automatically during initialization.

What do you exactly mean by power-up auto-read-phase pll-tuning function ?
Is it some MPR read functions ?
Or some other things ?

This is when I perform the MPR System Read Calibration and tune my PLL's read phase during power-up initialization.

[promach]

I suppose my current issue is not related to read phase alignment.
Could you advise about clk_serdes ?

[BrianHG]

You showed me a tIS violation.
You told me you were clocking Micron's 'ddr3.v' with your clk_serdes.

I'm trying to tell you you should have a proper generated DDR3_CK output from your design to feed Micron's 'ddr3.v' s CK input.

tIS is a setup timing relationship/clearance error between the CK and the command inputs.

If your FPGA's CK output is authentically true to clk_serdes as it is with all the command lines, then you have a tIS problem and it will also be a problem when you build an actual FPGA.

In fact, I know you have a tIS setup error visibly within you simulation waveform.

Now, there is a cheat to fix this, but, applying such a cheat is not proper form and it will come back to haunt you when you build an FPGA and you have not properly accommodated for a true CK output from the FPGA.  Either properly generate your true CK output or use the cheat.

Maybe searching for some other vendor's app notes on FPGA DDR3 implementations might help you out.  Lattice has some good info on how they generate their clocks, command, and latch data.  They kind of sit in-between Altera's old method and Xilinx current implementation.

[promach]

tIS is a setup timing relationship/clearance error between the CK and the command inputs.

Command inputs are driven using 87.5MHz clk_serdes , not 350MHz ck

[BrianHG]

  Just look at the picture...

[promach]

Ok, I have solved tIS violation with your hint/suggestion just above.

However, what is wrong with the following tMRD violation ?

[BrianHG]

We have covered this one already.
Go back around 22 pages.

[promach]

I think I know what is wrong now.  <-- no consecutive MRD command.

In this case, this means all command inputs need to be driven using 350MHz ck signal

[promach]

Why vivado simulator aborts (with fseek error) at line 665 of Micron simulation model ?

Code: [Select]

restart
INFO: [Simtcl 6-17] Simulation restarted
run 710 us
test_ddr3_memory_controller.mem.file_io_open: at time                    0 WARNING: no +model_data option specified, using /tmp.
test_ddr3_memory_controller.mem.open_bank_file: at time 0 INFO: opening /tmp/test_ddr3_memory_controller.mem.open_bank_file.0.
test_ddr3_memory_controller.mem.open_bank_file: at time 0 INFO: opening /tmp/test_ddr3_memory_controller.mem.open_bank_file.1.
test_ddr3_memory_controller.mem.open_bank_file: at time 0 INFO: opening /tmp/test_ddr3_memory_controller.mem.open_bank_file.2.
test_ddr3_memory_controller.mem.open_bank_file: at time 0 INFO: opening /tmp/test_ddr3_memory_controller.mem.open_bank_file.3.
test_ddr3_memory_controller.mem.open_bank_file: at time 0 INFO: opening /tmp/test_ddr3_memory_controller.mem.open_bank_file.4.
test_ddr3_memory_controller.mem.open_bank_file: at time 0 INFO: opening /tmp/test_ddr3_memory_controller.mem.open_bank_file.5.
test_ddr3_memory_controller.mem.open_bank_file: at time 0 INFO: opening /tmp/test_ddr3_memory_controller.mem.open_bank_file.6.
test_ddr3_memory_controller.mem.open_bank_file: at time 0 INFO: opening /tmp/test_ddr3_memory_controller.mem.open_bank_file.7.
test_ddr3_memory_controller.mem.cmd_task: at time 701901528.0 ps INFO: Load Mode 2
test_ddr3_memory_controller.mem.cmd_task: at time 701901528.0 ps INFO: Load Mode 2 Partial Array Self Refresh = Bank 0-7
test_ddr3_memory_controller.mem.cmd_task: at time 701901528.0 ps INFO: Load Mode 2 CAS Write Latency =           5
test_ddr3_memory_controller.mem.cmd_task: at time 701901528.0 ps INFO: Load Mode 2 Auto Self Refresh = Disabled
test_ddr3_memory_controller.mem.cmd_task: at time 701901528.0 ps INFO: Load Mode 2 Self Refresh Temperature = Normal
test_ddr3_memory_controller.mem.cmd_task: at time 701901528.0 ps INFO: Load Mode 2 Dynamic ODT = Disabled
test_ddr3_memory_controller.mem.cmd_task: at time 701915814.0 ps INFO: Load Mode 3
test_ddr3_memory_controller.mem.cmd_task: at time 701915814.0 ps INFO: Load Mode 3 MultiPurpose Register Select = Pre-defined pattern
test_ddr3_memory_controller.mem.cmd_task: at time 701915814.0 ps INFO: Load Mode 3 MultiPurpose Register Enable = Disabled
test_ddr3_memory_controller.mem.cmd_task: at time 701930100.0 ps INFO: Load Mode 1
test_ddr3_memory_controller.mem.cmd_task: at time 701930100.0 ps INFO: Load Mode 1 DLL Enable = Enabled
test_ddr3_memory_controller.mem.cmd_task: at time 701930100.0 ps INFO: Load Mode 1 Output Drive Strength =          34 Ohm
test_ddr3_memory_controller.mem.cmd_task: at time 701930100.0 ps INFO: Load Mode 1 ODT Rtt = Disabled
test_ddr3_memory_controller.mem.cmd_task: at time 701930100.0 ps INFO: Load Mode 1 Additive Latency = 0
test_ddr3_memory_controller.mem.cmd_task: at time 701930100.0 ps INFO: Load Mode 1 Write Levelization = Disabled
test_ddr3_memory_controller.mem.cmd_task: at time 701930100.0 ps INFO: Load Mode 1 TDQS Enable = Disabled
test_ddr3_memory_controller.mem.cmd_task: at time 701930100.0 ps INFO: Load Mode 1 Qoff = Enabled
test_ddr3_memory_controller.mem.cmd_task: at time 701944385.0 ps INFO: Load Mode 0
test_ddr3_memory_controller.mem.cmd_task: at time 701944385.0 ps INFO: Load Mode 0 Burst Length =  8
test_ddr3_memory_controller.mem.cmd_task: at time 701944385.0 ps INFO: Load Mode 0 Burst Order = Sequential
test_ddr3_memory_controller.mem.cmd_task: at time 701944385.0 ps INFO: Load Mode 0 CAS Latency =           5
test_ddr3_memory_controller.mem.cmd_task: at time 701944385.0 ps INFO: Load Mode 0 DLL Reset = Reset DLL
test_ddr3_memory_controller.mem.cmd_task: at time 701944385.0 ps INFO: Load Mode 0 Write Recovery =           5
test_ddr3_memory_controller.mem.cmd_task: at time 701944385.0 ps INFO: Load Mode 0 Power Down Mode = DLL on
test_ddr3_memory_controller.mem.cmd_task: at time 701981528.0 ps INFO: ZQ        long = 1
test_ddr3_memory_controller.mem.cmd_task: at time 701981528.0 ps INFO: Initialization Sequence is complete
test_ddr3_memory_controller.mem.main: at time 703412957.0 ps ERROR: Write Recovery =           5 is illegal @tCK(avg) = 2857.144531
test_ddr3_memory_controller.mem.cmd_task: at time 703452957.0 ps INFO: Precharge All
test_ddr3_memory_controller.mem.cmd_task: at time 703455814.0 ps INFO: Precharge All
test_ddr3_memory_controller.mem.cmd_task: at time 703458671.0 ps INFO: Precharge All
test_ddr3_memory_controller.mem.cmd_task: at time 703461528.0 ps INFO: Precharge All
test_ddr3_memory_controller.mem.cmd_task: at time 703464385.0 ps INFO: Precharge All
test_ddr3_memory_controller.mem.cmd_task: at time 703467242.0 ps INFO: Load Mode 3
test_ddr3_memory_controller.mem.cmd_task: at time 703467242.0 ps INFO: Load Mode 3 MultiPurpose Register Select = Pre-defined pattern
test_ddr3_memory_controller.mem.cmd_task: at time 703467242.0 ps INFO: Load Mode 3 MultiPurpose Register Enable = Enabled
test_ddr3_memory_controller.mem.cmd_task: at time 703504385.0 ps INFO: Read      bank 3 col 000, auto precharge 0
test_ddr3_memory_controller.mem.read_from_file: at time 703517242.0 ps ERROR: fseek to           x failed
$finish called at time : 703517242 ps : File "/home/phung/Downloads/DDR_backup/DDR_Xilinx_Vivado/DDR_Xilinx_Vivado.srcs/sources_1/imports/DDR/ddr3.v" Line 665
run: Time (s): cpu = 00:00:16 ; elapsed = 00:01:39 . Memory (MB): peak = 7895.703 ; gain = 0.000 ; free physical = 1212 ; free virtual = 7871


[promach]

Ok, the culprit seems to comes from the following line of log, but how to get around it ?

Code: [Select]

test_ddr3_memory_controller.mem.main: at time 703412957.0 ps ERROR: Write Recovery =           5 is illegal @tCK(avg) = 2857.144531

[promach]

ok, why does the Micron simulation model complains about Write Recovery = 5 is illegal before the fseek error comes into place ?

[promach]

When the command inputs are driven using 350MHz ck_270 signal, I have a lot of setup timing violations.

Is there any other way to go around the tMRD violations without resorting to the use of 350MHz ck_270 signal ?

[BrianHG]

Shouldn't the commands be on the ck_0 signal?
What's your command synchronization length?

I have 1 on my CK_0/2 plus 2 on my ck_0 side, then that feeds the DDR Output buffers.

It's beginning to look like Xilinx is not much faster is any than Altera except for the peak throughput of their DDR-IO pin buffers.

Looking at Lattice, it is the same story, but a little worse as their DDR buffers cannot achieve full speed unless you use them in DDRX2 mode, which is actually a QDR mode, IE 4:1 serdes.

[promach]

Shouldn't the commands be on the ck_0 signal?
What's your command synchronization length?

ck_270 is to generate 90 degrees phase LEAD relative to ck, since the command inputs bits need to be sampled at their middlemost bit position.

Those setup timing violations are not related to FF synchronizers length.

The logic inside the ck_270 is just a bit large.

[BrianHG]

All the command, controls and address are sampled right on the rise of CK_0, not the read position.

Only the DDR3 reading the DQ and DM sample on the CK_90 and CK_270 positions.

[promach]

no, check the initialization sequence you sent earlier.

all command inputs are sent to DDR3 RAM with a 90 degree phase difference with respect to ck signal.

And there is no phase difference between DQ and DQS for write operation.

[promach]

And almost all the setup timing violations within ck_270 domain is due to multi-level comparison hardware for wait_count signal.

I think I will halve the bitwidth of wait_count, and set up another counter tracking variable to sequentially increment wait_count with multiple stages

[BrianHG]

no, check the initialization sequence you sent earlier.

all command inputs are sent to DDR3 RAM with a 90 degree phase difference with respect to ck signal.

And there is no phase difference between DQ and DQS for write operation.

Show me where and I will prove you wrong.

[promach]

See https://www.eevblog.com/forum/fpga/ddr3-initialization-sequence-issue/msg3707713/#msg3707713

[BrianHG]

My photo showing Micron's example 'B' has the command's valid point dead center on the rising of the DDR3 CK.
It also has the don't care dead center over the inverted CK#.

tIS and tIH (not shown) is just the bare minimum setup and tIH hold time, not the best case scenario.

Now, to get the maximum tIS and tIH out of the FPGA outputs, you would target the transition of all command and address lines on the CK# transition.  This is not ck_270.

I get the feeling you have never designed a synchronous bus interface before.

Take a look at my old Modelsim snapshots.  They demonstrate this exactly.