Author Topic: DDR3 initialization sequence issue  (Read 49036 times)

0 Members and 1 Guest are viewing this topic.

Offline BrianHG

  • Super Contributor
  • ***
  • Posts: 7189
  • Country: ca
Re: DDR3 initialization sequence issue
« Reply #550 on: September 23, 2021, 10:06:24 pm »
tIS is measured with reference to your DDR_CK output.
Where is your DDR_CK signal in that simulation?

If you are doing a true Gate-Level timing simulation, the output and input timing will not match infinite and perfect IO speed of your old RTL simulations within Modelsim depending on the way you programed your IOs.

Good luck.

Note that in my sims, I was using Altera's authentic DDR-IO buffers & PLL to generate the outputs to feed Micron's DDR3 model.  I was not using a home-made simple logic DDR buffer or software generated PLL.
« Last Edit: September 23, 2021, 10:15:34 pm by BrianHG »
 
The following users thanked this post: promach

Offline promach

  • Frequent Contributor
  • **
  • Posts: 875
  • Country: us
Re: DDR3 initialization sequence issue
« Reply #551 on: September 24, 2021, 02:47:02 am »
In this case, how would I go around the IO timing issue resulted from the use of vendor's DDR-IO buffers and PLL primitives ?

Note: I am using clk_serdes from PLL to drive CKE , RAS_N , and CAS_N signals
« Last Edit: September 24, 2021, 02:52:29 am by promach »
 

Offline BrianHG

  • Super Contributor
  • ***
  • Posts: 7189
  • Country: ca
Re: DDR3 initialization sequence issue
« Reply #552 on: September 24, 2021, 03:38:26 am »
In Quartus, the test bench simulations are a separate entity from a complete FPGA project.
I would normally design my FPGA and do a full compile then run a gate-level timing simulation.

My top_tb.v is a separate entity which only has access to the IO pins of the FPGA project.

The actual IO pins which are wired on my PCB to the DDR3 ram chip are what are wired to Micron's DDR3 model.  This is the only way you can see the true timing at the IO pins in the DDR3 controller.

In my RTL sims, I am doing the same thing for just the DDR3 IOs, however, gate-level timing sims does alter the timing of the read data.  Though, my power-up auto-read-phase pll-tuning function is wide enough to catch and correct for this automatically during initialization.

This method offers good true timing sims with ModelSim with the older FPGA like Cyclone IV and earlier.  This method is now deprecated with the latest generation of FPGAs from Intel for strict board-timing analysis tools which no longer support this type of simulation.

I do not know how this is done in Xilinx.
 
The following users thanked this post: promach

Offline BrianHG

  • Super Contributor
  • ***
  • Posts: 7189
  • Country: ca
Re: DDR3 initialization sequence issue
« Reply #553 on: September 24, 2021, 03:42:10 am »
Remember, for true timing sims, you need to compile an actual FPGA as that timing is affected by things like the FPGA grade, IO voltage standard, and which pins you are actually using and where they are located on the FPGA.

This true timing information just isn't available to Modelsim under a normal logic RTL compile.
 
The following users thanked this post: promach

Offline promach

  • Frequent Contributor
  • **
  • Posts: 875
  • Country: us
Re: DDR3 initialization sequence issue
« Reply #554 on: September 24, 2021, 03:57:41 am »
Quote
however, gate-level timing sims does alter the timing of the read data.  Though, my power-up auto-read-phase pll-tuning function is wide enough to catch and correct for this automatically during initialization.

What do you exactly mean by power-up auto-read-phase pll-tuning function ?
Is it some MPR read functions ?
Or some other things ?
 

Offline BrianHG

  • Super Contributor
  • ***
  • Posts: 7189
  • Country: ca
Re: DDR3 initialization sequence issue
« Reply #555 on: September 24, 2021, 04:05:01 am »
Quote
however, gate-level timing sims does alter the timing of the read data.  Though, my power-up auto-read-phase pll-tuning function is wide enough to catch and correct for this automatically during initialization.

What do you exactly mean by power-up auto-read-phase pll-tuning function ?
Is it some MPR read functions ?
Or some other things ?
This is when I perform the MPR System Read Calibration and tune my PLL's read phase during power-up initialization.
 

Offline promach

  • Frequent Contributor
  • **
  • Posts: 875
  • Country: us
Re: DDR3 initialization sequence issue
« Reply #556 on: September 24, 2021, 07:06:06 am »
I suppose my current issue is not related to read phase alignment.
Could you advise about clk_serdes ?

 

Offline BrianHG

  • Super Contributor
  • ***
  • Posts: 7189
  • Country: ca
Re: DDR3 initialization sequence issue
« Reply #557 on: September 24, 2021, 07:20:29 am »
You showed me a tIS violation.
You told me you were clocking Micron's 'ddr3.v' with your clk_serdes.

I'm trying to tell you you should have a proper generated DDR3_CK output from your design to feed Micron's 'ddr3.v' s CK input.

tIS is a setup timing relationship/clearance error between the CK and the command inputs.

If your FPGA's CK output is authentically true to clk_serdes as it is with all the command lines, then you have a tIS problem and it will also be a problem when you build an actual FPGA.

In fact, I know you have a tIS setup error visibly within you simulation waveform.

Now, there is a cheat to fix this, but, applying such a cheat is not proper form and it will come back to haunt you when you build an FPGA and you have not properly accommodated for a true CK output from the FPGA.  Either properly generate your true CK output or use the cheat.

Maybe searching for some other vendor's app notes on FPGA DDR3 implementations might help you out.  Lattice has some good info on how they generate their clocks, command, and latch data.  They kind of sit in-between Altera's old method and Xilinx current implementation.
 
The following users thanked this post: promach

Offline promach

  • Frequent Contributor
  • **
  • Posts: 875
  • Country: us
Re: DDR3 initialization sequence issue
« Reply #558 on: September 24, 2021, 07:27:10 am »
Quote
tIS is a setup timing relationship/clearance error between the CK and the command inputs.

Command inputs are driven using 87.5MHz clk_serdes , not 350MHz ck
 

Offline BrianHG

  • Super Contributor
  • ***
  • Posts: 7189
  • Country: ca
Re: DDR3 initialization sequence issue
« Reply #559 on: September 24, 2021, 07:38:11 am »
 :palm: Just look at the picture...
 
The following users thanked this post: promach

Offline promach

  • Frequent Contributor
  • **
  • Posts: 875
  • Country: us
Re: DDR3 initialization sequence issue
« Reply #560 on: September 24, 2021, 08:13:44 am »
Ok, I have solved tIS violation with your hint/suggestion just above.

However, what is wrong with the following tMRD violation ?

 

Offline BrianHG

  • Super Contributor
  • ***
  • Posts: 7189
  • Country: ca
Re: DDR3 initialization sequence issue
« Reply #561 on: September 24, 2021, 08:16:27 am »
We have covered this one already.
Go back around 22 pages.
 

Offline promach

  • Frequent Contributor
  • **
  • Posts: 875
  • Country: us
Re: DDR3 initialization sequence issue
« Reply #562 on: September 24, 2021, 08:18:16 am »
I think I know what is wrong now.  <-- no consecutive MRD command.

In this case, this means all command inputs need to be driven using 350MHz ck signal
 

Offline promach

  • Frequent Contributor
  • **
  • Posts: 875
  • Country: us
Re: DDR3 initialization sequence issue
« Reply #563 on: September 24, 2021, 09:52:13 am »
Why vivado simulator aborts (with fseek error) at line 665 of Micron simulation model ?

Code: [Select]
restart
INFO: [Simtcl 6-17] Simulation restarted
run 710 us
test_ddr3_memory_controller.mem.file_io_open: at time                    0 WARNING: no +model_data option specified, using /tmp.
test_ddr3_memory_controller.mem.open_bank_file: at time 0 INFO: opening /tmp/test_ddr3_memory_controller.mem.open_bank_file.0.
test_ddr3_memory_controller.mem.open_bank_file: at time 0 INFO: opening /tmp/test_ddr3_memory_controller.mem.open_bank_file.1.
test_ddr3_memory_controller.mem.open_bank_file: at time 0 INFO: opening /tmp/test_ddr3_memory_controller.mem.open_bank_file.2.
test_ddr3_memory_controller.mem.open_bank_file: at time 0 INFO: opening /tmp/test_ddr3_memory_controller.mem.open_bank_file.3.
test_ddr3_memory_controller.mem.open_bank_file: at time 0 INFO: opening /tmp/test_ddr3_memory_controller.mem.open_bank_file.4.
test_ddr3_memory_controller.mem.open_bank_file: at time 0 INFO: opening /tmp/test_ddr3_memory_controller.mem.open_bank_file.5.
test_ddr3_memory_controller.mem.open_bank_file: at time 0 INFO: opening /tmp/test_ddr3_memory_controller.mem.open_bank_file.6.
test_ddr3_memory_controller.mem.open_bank_file: at time 0 INFO: opening /tmp/test_ddr3_memory_controller.mem.open_bank_file.7.
test_ddr3_memory_controller.mem.cmd_task: at time 701901528.0 ps INFO: Load Mode 2
test_ddr3_memory_controller.mem.cmd_task: at time 701901528.0 ps INFO: Load Mode 2 Partial Array Self Refresh = Bank 0-7
test_ddr3_memory_controller.mem.cmd_task: at time 701901528.0 ps INFO: Load Mode 2 CAS Write Latency =           5
test_ddr3_memory_controller.mem.cmd_task: at time 701901528.0 ps INFO: Load Mode 2 Auto Self Refresh = Disabled
test_ddr3_memory_controller.mem.cmd_task: at time 701901528.0 ps INFO: Load Mode 2 Self Refresh Temperature = Normal
test_ddr3_memory_controller.mem.cmd_task: at time 701901528.0 ps INFO: Load Mode 2 Dynamic ODT = Disabled
test_ddr3_memory_controller.mem.cmd_task: at time 701915814.0 ps INFO: Load Mode 3
test_ddr3_memory_controller.mem.cmd_task: at time 701915814.0 ps INFO: Load Mode 3 MultiPurpose Register Select = Pre-defined pattern
test_ddr3_memory_controller.mem.cmd_task: at time 701915814.0 ps INFO: Load Mode 3 MultiPurpose Register Enable = Disabled
test_ddr3_memory_controller.mem.cmd_task: at time 701930100.0 ps INFO: Load Mode 1
test_ddr3_memory_controller.mem.cmd_task: at time 701930100.0 ps INFO: Load Mode 1 DLL Enable = Enabled
test_ddr3_memory_controller.mem.cmd_task: at time 701930100.0 ps INFO: Load Mode 1 Output Drive Strength =          34 Ohm
test_ddr3_memory_controller.mem.cmd_task: at time 701930100.0 ps INFO: Load Mode 1 ODT Rtt = Disabled
test_ddr3_memory_controller.mem.cmd_task: at time 701930100.0 ps INFO: Load Mode 1 Additive Latency = 0
test_ddr3_memory_controller.mem.cmd_task: at time 701930100.0 ps INFO: Load Mode 1 Write Levelization = Disabled
test_ddr3_memory_controller.mem.cmd_task: at time 701930100.0 ps INFO: Load Mode 1 TDQS Enable = Disabled
test_ddr3_memory_controller.mem.cmd_task: at time 701930100.0 ps INFO: Load Mode 1 Qoff = Enabled
test_ddr3_memory_controller.mem.cmd_task: at time 701944385.0 ps INFO: Load Mode 0
test_ddr3_memory_controller.mem.cmd_task: at time 701944385.0 ps INFO: Load Mode 0 Burst Length =  8
test_ddr3_memory_controller.mem.cmd_task: at time 701944385.0 ps INFO: Load Mode 0 Burst Order = Sequential
test_ddr3_memory_controller.mem.cmd_task: at time 701944385.0 ps INFO: Load Mode 0 CAS Latency =           5
test_ddr3_memory_controller.mem.cmd_task: at time 701944385.0 ps INFO: Load Mode 0 DLL Reset = Reset DLL
test_ddr3_memory_controller.mem.cmd_task: at time 701944385.0 ps INFO: Load Mode 0 Write Recovery =           5
test_ddr3_memory_controller.mem.cmd_task: at time 701944385.0 ps INFO: Load Mode 0 Power Down Mode = DLL on
test_ddr3_memory_controller.mem.cmd_task: at time 701981528.0 ps INFO: ZQ        long = 1
test_ddr3_memory_controller.mem.cmd_task: at time 701981528.0 ps INFO: Initialization Sequence is complete
test_ddr3_memory_controller.mem.main: at time 703412957.0 ps ERROR: Write Recovery =           5 is illegal @tCK(avg) = 2857.144531
test_ddr3_memory_controller.mem.cmd_task: at time 703452957.0 ps INFO: Precharge All
test_ddr3_memory_controller.mem.cmd_task: at time 703455814.0 ps INFO: Precharge All
test_ddr3_memory_controller.mem.cmd_task: at time 703458671.0 ps INFO: Precharge All
test_ddr3_memory_controller.mem.cmd_task: at time 703461528.0 ps INFO: Precharge All
test_ddr3_memory_controller.mem.cmd_task: at time 703464385.0 ps INFO: Precharge All
test_ddr3_memory_controller.mem.cmd_task: at time 703467242.0 ps INFO: Load Mode 3
test_ddr3_memory_controller.mem.cmd_task: at time 703467242.0 ps INFO: Load Mode 3 MultiPurpose Register Select = Pre-defined pattern
test_ddr3_memory_controller.mem.cmd_task: at time 703467242.0 ps INFO: Load Mode 3 MultiPurpose Register Enable = Enabled
test_ddr3_memory_controller.mem.cmd_task: at time 703504385.0 ps INFO: Read      bank 3 col 000, auto precharge 0
test_ddr3_memory_controller.mem.read_from_file: at time 703517242.0 ps ERROR: fseek to           x failed
$finish called at time : 703517242 ps : File "/home/phung/Downloads/DDR_backup/DDR_Xilinx_Vivado/DDR_Xilinx_Vivado.srcs/sources_1/imports/DDR/ddr3.v" Line 665
run: Time (s): cpu = 00:00:16 ; elapsed = 00:01:39 . Memory (MB): peak = 7895.703 ; gain = 0.000 ; free physical = 1212 ; free virtual = 7871
 

Offline promach

  • Frequent Contributor
  • **
  • Posts: 875
  • Country: us
Re: DDR3 initialization sequence issue
« Reply #564 on: September 24, 2021, 09:56:59 am »
Ok, the culprit seems to comes from the following line of log, but how to get around it ?

Code: [Select]
test_ddr3_memory_controller.mem.main: at time 703412957.0 ps ERROR: Write Recovery =           5 is illegal @tCK(avg) = 2857.144531
 

Offline promach

  • Frequent Contributor
  • **
  • Posts: 875
  • Country: us
Re: DDR3 initialization sequence issue
« Reply #565 on: September 24, 2021, 10:45:46 am »
ok, why does the Micron simulation model complains about Write Recovery = 5 is illegal before the fseek error comes into place ?
 

Offline promach

  • Frequent Contributor
  • **
  • Posts: 875
  • Country: us
Re: DDR3 initialization sequence issue
« Reply #566 on: September 24, 2021, 11:29:19 am »
When the command inputs are driven using 350MHz ck_270 signal, I have a lot of setup timing violations.

Is there any other way to go around the tMRD violations without resorting to the use of 350MHz ck_270 signal ?

 

Offline BrianHG

  • Super Contributor
  • ***
  • Posts: 7189
  • Country: ca
Re: DDR3 initialization sequence issue
« Reply #567 on: September 24, 2021, 10:29:40 pm »
Shouldn't the commands be on the ck_0 signal?
What's your command synchronization length?

I have 1 on my CK_0/2 plus 2 on my ck_0 side, then that feeds the DDR Output buffers.

It's beginning to look like Xilinx is not much faster is any than Altera except for the peak throughput of their DDR-IO pin buffers.

Looking at Lattice, it is the same story, but a little worse as their DDR buffers cannot achieve full speed unless you use them in DDRX2 mode, which is actually a QDR mode, IE 4:1 serdes.
 

Offline promach

  • Frequent Contributor
  • **
  • Posts: 875
  • Country: us
Re: DDR3 initialization sequence issue
« Reply #568 on: September 25, 2021, 01:13:42 am »
Quote
Shouldn't the commands be on the ck_0 signal?
What's your command synchronization length?

ck_270 is to generate 90 degrees phase LEAD relative to ck, since the command inputs bits need to be sampled at their middlemost bit position.

Those setup timing violations are not related to FF synchronizers length.

The logic inside the ck_270 is just a bit large.



 

Offline BrianHG

  • Super Contributor
  • ***
  • Posts: 7189
  • Country: ca
Re: DDR3 initialization sequence issue
« Reply #569 on: September 25, 2021, 02:03:58 am »
All the command, controls and address are sampled right on the rise of CK_0, not the read position.

Only the DDR3 reading the DQ and DM sample on the CK_90 and CK_270 positions.
 

Offline promach

  • Frequent Contributor
  • **
  • Posts: 875
  • Country: us
Re: DDR3 initialization sequence issue
« Reply #570 on: September 25, 2021, 02:54:22 am »
no, check the initialization sequence you sent earlier.

all command inputs are sent to DDR3 RAM with a 90 degree phase difference with respect to ck signal.

And there is no phase difference between DQ and DQS for write operation.
 

Offline promach

  • Frequent Contributor
  • **
  • Posts: 875
  • Country: us
Re: DDR3 initialization sequence issue
« Reply #571 on: September 25, 2021, 02:57:49 am »
And almost all the setup timing violations within ck_270 domain is due to multi-level comparison hardware for wait_count signal.

I think I will halve the bitwidth of wait_count, and set up another counter tracking variable to sequentially increment wait_count with multiple stages
 

Offline BrianHG

  • Super Contributor
  • ***
  • Posts: 7189
  • Country: ca
Re: DDR3 initialization sequence issue
« Reply #572 on: September 25, 2021, 03:37:59 am »
no, check the initialization sequence you sent earlier.

all command inputs are sent to DDR3 RAM with a 90 degree phase difference with respect to ck signal.

And there is no phase difference between DQ and DQS for write operation.
Show me where and I will prove you wrong.
 

Offline promach

  • Frequent Contributor
  • **
  • Posts: 875
  • Country: us
 

Offline BrianHG

  • Super Contributor
  • ***
  • Posts: 7189
  • Country: ca
Re: DDR3 initialization sequence issue
« Reply #574 on: September 25, 2021, 03:52:43 am »
My photo showing Micron's example 'B' has the command's valid point dead center on the rising of the DDR3 CK.
It also has the don't care dead center over the inverted CK#.

tIS and tIH (not shown) is just the bare minimum setup and tIH hold time, not the best case scenario.

Now, to get the maximum tIS and tIH out of the FPGA outputs, you would target the transition of all command and address lines on the CK# transition.  This is not ck_270.

I get the feeling you have never designed a synchronous bus interface before.

Take a look at my old Modelsim snapshots.  They demonstrate this exactly.
« Last Edit: September 25, 2021, 03:54:24 am by BrianHG »
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf