DDR3 initialization sequence issue

[promach]

The C0 pin of IDDR must be connected to the same clock as C0 pin of the corresponding ODDR. The C1 pin of IDDR must be connected to the same clock as C1 pin of the corresponding ODDR.

How to re-purpose IDDR and ODDR for PLL dynamic phase shift since incoming read DQS needs to be tuned to 90 degree apart from read DQ bits ?

[NorthGuy]

The C0 pin of IDDR must be connected to the same clock as C0 pin of the corresponding ODDR. The C1 pin of IDDR must be connected to the same clock as C1 pin of the corresponding ODDR.

How to re-purpose IDDR and ODDR for PLL dynamic phase shift since incoming read DQS needs to be tuned to 90 degree apart from read DQ bits ?

You can either re-adjust PLL phase dynamically, so it's either tuned for inputs or for outputs (but not both at the same time). This is probably too long a process for quick read-to-write and write-to-read transitions.

Or you can use a clock switcher. There must be one in Spartan-6, but I'm not sure. Bbut your reads and writes still will be in different clock domains.

Or you can insert IDELAY into DQS and DQ paths and adjust them so that they're properly aligned to CK.

Or you can switch to 7-series.

[promach]

You can either re-adjust PLL phase dynamically, so it's either tuned for inputs or for outputs (but not both at the same time). This is probably too long a process for quick read-to-write and write-to-read transitions.

I do not understand what you exactly meant by "re-adjust" ?

Or you can use a clock switcher. There must be one in Spartan-6, but I'm not sure. Bbut your reads and writes still will be in different clock domains.

But clock switcher still need to be connected to IDDR and ODDR, which means the issue will still be there.

Or you can insert IDELAY into DQS and DQ paths and adjust them so that they're properly aligned to CK.

I had tried using IDELAY, but it caused a lot more routing error than PLL dynamic phase shift approach

[NorthGuy]

You can either re-adjust PLL phase dynamically, so it's either tuned for inputs or for outputs (but not both at the same time). This is probably too long a process for quick read-to-write and write-to-read transitions.

I do not understand what you exactly meant by "re-adjust" ?

I mean changing phase.

Or you can use a clock switcher. There must be one in Spartan-6, but I'm not sure. Bbut your reads and writes still will be in different clock domains.

But clock switcher still need to be connected to IDDR and ODDR, which means the issue will still be there.

The output of clock switcher would feed BUFG which then feeds IDDR and ODDR. This is not a very good solution though.

Or you can insert IDELAY into DQS and DQ paths and adjust them so that they're properly aligned to CK.

I had tried using IDELAY, but it caused a lot more routing error than PLL dynamic phase shift approach

You probably connected something wrong.

[promach]

I mean changing phase.

How would this solution exactly help to eliminate the IODDR same clock domain restriction ?

The output of clock switcher would feed BUFG which then feeds IDDR and ODDR. This is not a very good solution though.

This is a simple, straightforward solution (need to verify first), but I am not sure why you said that it is not a very good solution ?

You probably connected something wrong.

Not really. I had talked to litedram author and he is using PLL dynamic phase shift for spartan-6.
Not only that, IODELAY primitive on spartan-6 has some internal hardware design issue that would cause data bit  loss.  So, not recommended to use IODELAY in this case.

[NorthGuy]

I mean changing phase.

How would this solution exactly help to eliminate the IODDR same clock domain restriction ?

read - change phase for wrtie - write - change phase for read - read etc.

This will make things very slow.

The output of clock switcher would feed BUFG which then feeds IDDR and ODDR. This is not a very good solution though.

This is a simple, straightforward solution (need to verify first), but I am not sure why you said that it is not a very good solution ?

I think it may be difficult to deal with passing data between the switching clock domain and others. May be not. You need to evaluate this.

Not really. I had talked to litedram author and he is using PLL dynamic phase shift for spartan-6.

For DDR3? You can ask him how he dealt with the IDDR/ODDR clocking restrictions. May be there are some tricks we don't know about. Then evaluate if you can do the same.

Not only that, IODELAY primitive on spartan-6 has some internal hardware design issue that would cause data bit  loss.  So, not recommended to use IODELAY in this case.

You mean this: https://www.xilinx.com/support/answers/41083.html

The problem is only at higher data rates. Look up your Spartan in their table. May not apply to you.

[promach]

read - change phase for wrtie - write - change phase for read - read etc.

This will make things very slow.

Wait, how is changing phase able to get around same clock (similar frequency and phase shift) restriction of IODDR ?

I think it may be difficult to deal with passing data between the switching clock domain and others. May be not. You need to evaluate this.

I am confuse whether to use BUFGMUX primitive or BUFGMUX_1 primitive.

For DDR3? You can ask him how he dealt with the IDDR/ODDR clocking restrictions. May be there are some tricks we don't know about. Then evaluate if you can do the same.

Yes, for DDR3 on spartan-6.

You mean this: https://www.xilinx.com/support/answers/41083.html

The problem is only at higher data rates. Look up your Spartan in their table. May not apply to you.

Does that value of the maximum data rate Mb/s apply to x16 (16 bits) ?

By the way, litedram author mentioned that fine delay was done using fixed phase shift of the clocks with the PLL and coarse delays with the bitslip.

[NorthGuy]

read - change phase for wrtie - write - change phase for read - read etc.

This will make things very slow.

Wait, how is changing phase able to get around same clock (similar frequency and phase shift) restriction of IODDR?

Same as clock switching, except instead of switching you change phase of a single clock.

Does that value of the maximum data rate Mb/s apply to x16 (16 bits) ?

These are single pin rates - clock multiplied by x2.

By the way, litedram author mentioned that fine delay was done using fixed phase shift of the clocks with the PLL and coarse delays with the bitslip.

This means that he somehow bypassed the single clock restriction for IDDR/ODDR, ask him how.

[promach]

I am still waiting for litedram's author reply.

I have linked to the wrong litedram file in previous post.

See https://github.com/enjoy-digital/litedram/blob/master/litedram/phy/s6ddrphy.py instead for DDR3 PHY for litedram on spartan-6

[NorthGuy]

I am still waiting for litedram's author reply.

I have linked to the wrong litedram file in previous post.

See https://github.com/enjoy-digital/litedram/blob/master/litedram/phy/s6ddrphy.py instead for DDR3 PHY for litedram on spartan-6

It's python I guess they use it to generate HDL

Looks like they use different clocks - sdram_full_rd_clk and sdram_full_wr_clk

[NorthGuy]

Also their SERDES blocks are in "SDR" mode - go figure.

[promach]

I am also using serdes in SDR mode for DDR3 controller. There is nothing wrong with this.

Please correct me if wrong.

[NorthGuy]

I am also using serdes in SDR mode for DDR3 controller. There is nothing wrong with this.

Please correct me if wrong.

I thought you use IDDR/ODDR in place of ISERDES/OSERDES.

As far as can understand thir python, they use OSERDESE2 in "SDR" mode which produces SDR signal on the output pin. You need DDR for DDR3. Or you will write every second bit only - the odd bits, which has to be set on the falling edge will always be the same as the corresponding even bit. If you do this, this will use only half of the memory.

[promach]

For DDR3 controller, I am using BOTH Xilinx IODDR , and home-made IOSERDES.

Note: It is your suggestion to use TWO home-made serializers and deserializers on SDR mode to work for DDR mode.

[NorthGuy]

For DDR3 controller, I am using BOTH Xilinx IODDR , and home-made IOSERDES.

I misunderstood what you said. To me, all-capital letters refer to built-in blocks. Like you use home-made serdes (serializers), but they use Xilinx's ISERDES/OSERDES.

The difference is that yours are in fabric and you can connect them as you wish, so you connect them to ODDR/IDDR do deal with DDR signals.

But theirs are built-in and can only be connected directly to IOBUF and hence must deal with DDR signals. However, they seem to use "SDR" mode, which will miss all the bits on falling edges. I have no idea how this can possibly work. You can ask them this.

<edit>Actually, SDR may, sort of, work, but it will access only half of the memory (miss every second bit), and will achieve only half of the full bandwidth. Is that what they're doing?

[promach]

Is it possible at all to re-code my own homemade IODDR in order to get around the same clock restriction, hence the routing issue ?

[promach]

I have solved the routing conflict for IODDR2 (and solved the setup timing issue for dq_r_q0 signal) by using my own homemade IDDR,  I will replace all of the existing Xilinx IODDR2 primitive with my own homemade version later.

Now I am solving the last few setup timing violation

Any idea about the following highlighted path on ODDR2_ldqs_iobuf_en ?

Code: [Select]

// see [url]https://www.xilinx.com/support/documentation/user_guides/ug381.pdf#page=61[/url]
// 'data_read_is_ongoing' signal is not of double-data-rate signals,
// but it is connected to T port of IOBUF where its I port is fed in with double-data-rate DQS signals,
// thus the purpose of having the following ODDR2 primitives

ODDR2 #(
.DDR_ALIGNMENT("C0"),  // Sets output alignment to "NONE", "C0" or "C1"
.INIT(1'b0),  // Sets initial state of the Q output to 1'b0 or 1'b1
.SRTYPE("ASYNC")  // Specifies "SYNC" or "ASYNC" set/reset
)
ODDR2_ldqs_iobuf_en(
.Q(ldqs_iobuf_enable),  // 1-bit DDR output data
.C0(ck_90),  // 1-bit clock input
.C1(ck_90),  // 1-bit clock input
.CE(1'b1),  // 1-bit clock enable input
.D0(data_read_is_ongoing),    // 1-bit DDR data input (associated with C0)
.D1(data_read_is_ongoing),    // 1-bit DDR data input (associated with C1)
.R(1'b0),    // 1-bit reset input
.S(1'b0)     // 1-bit set input
);

[NorthGuy]

I have solved the routing conflict for IODDR2 (and solved the setup timing issue for dq_r_q0 signal) by using my own homemade IDDR,  I will replace all of the existing Xilinx IODDR2 primitive with my own homemade version later.

Yes, custom iddr may work. The only problem is that the receiving flops won't be in IOB, so there will be some extra jitter.

Now I am solving the last few setup timing violation

You cannot pass the signal from ck to ck_90 - that's too fast. You need to solve this problem somehow. For example, you can pass from ck to ck_270 rather easily (e.g. synchronizing the ODDR with C1). Or, you can use an async FIFO to pass all your data from ck to ck_90. Or, you can ask @BrianHG how he did it.

[promach]

You cannot pass the signal from ck to ck_90 - that's too fast. You need to solve this problem somehow. For example, you can pass from ck to ck_270 rather easily (e.g. synchronizing the ODDR with C1). Or, you can use an async FIFO to pass all your data from ck to ck_90. Or, you can ask @BrianHG how he did it.

Would Multi-cycle path (MCP) formulation with feedback acknowledge help ?

[BrianHG]

I have solved the routing conflict for IODDR2 (and solved the setup timing issue for dq_r_q0 signal) by using my own homemade IDDR,  I will replace all of the existing Xilinx IODDR2 primitive with my own homemade version later.

Yes, custom iddr may work. The only problem is that the receiving flops won't be in IOB, so there will be some extra jitter.

Now I am solving the last few setup timing violation

You cannot pass the signal from ck to ck_90 - that's too fast. You need to solve this problem somehow. For example, you can pass from ck to ck_270 rather easily (e.g. synchronizing the ODDR with C1). Or, you can use an async FIFO to pass all your data from ck to ck_90. Or, you can ask @BrianHG how he did it.

I used a huge synchronization chain forced into logic cells away from auto-generated ram blocks allowing the compiler to re-time the logic cells along that chain to gracefully convert the data from from the ck_0 domain into the ck_90 domain, then once the data is in the ck_90 domain, it then feeds the DDR primitive's data input to drive the pin's output.

If I did not do so, I end up with either huge negative slack feeding the DDR buffer, or, a horrible ck_0 FMAX limit due to hold restrictions depending on the path the compiler chose to optimize my design.

My sync chain also holds in parallel the DM and the OE for the DQ.

[promach]

I used a huge synchronization chain forced into logic cells away from auto-generated ram blocks allowing the compiler to re-time the logic cells along that chain to gracefully convert the data from from the ck_0 domain into the ck_90 domain, then once the data is in the ck_90 domain, it then feeds the DDR primitive's data input to drive the pin's output.

What do you exactly mean by huge synchronization chain ?

My sync chain also holds in parallel the DM and the OE for the DQ.

How do you make sure that the vendor tool will actually implement (and route) the sync chain to be in parallel with control signals for the DQ ?

[BrianHG]

I mean that on the core ck_0, I have the:

{wd_h, wd_l, wd_oe, wm_h, wm_l} ready every clock.

on every ck_90, I have:

Code: [Select]

before_DDR_BUFFERS [4] <= {wd_h, wd_l, wd_oe, wm_h, wm_l};
before_DDR_BUFFERS [3] <= before_DDR_BUFFERS [4] ;
before_DDR_BUFFERS [2] <= before_DDR_BUFFERS [3] ;
before_DDR_BUFFERS [1] <= before_DDR_BUFFERS [2] ;
before_DDR_BUFFERS [0] <= before_DDR_BUFFERS [1] ;

DDR_BUFFERS_OUT <= before_DDR_BUFFERS [0] ;
This adds a serial chain of 6 clocks inside the ck_90 domain allowing the fitter to skew the clock timing of each step in that chain to allow error free data to shift it's ck_0 phase to ck_90 phase before it reaches the IO buffer's input DFF.  With a chain this size, this also means the source data {wd_h, wd_l, wd_oe, wm_h, wm_l} needs to be ready 6 clocks in advance.

Obviously, DDR_BUFFERS_OUT would be the DQ IODDR receiving the {wd_h, wd_l} as data input and the {wd_oe} drives the OE for those DQ buffers while the DM DDROUT receives the {wm_h, wm_l}.

See my source code 'BrianHG_DDR3_IO_PORT_ALTERA.sv' V1.00 beginning at line 585 until the end of the code.  You will also see I have a parameter option WDQ_CLK_270 to use a 270 degree write clock in place of the 90 degree which needs to swap the _h & _l, plus advance the _l by 1 write clock.

[BrianHG]

Quartus is smart and tries to move my above logic chain into a ram block.  However, ram blocks cannot shift time at every step along the way, only at the input or the output.  Also, ram blocks have a hard FMAX of 250MHz on the cheap small Cyclones/MAX 10s I made this ram controller to work on.  So I have used a keyword in declaring the logic cells ' before_DDR_BUFFERS [ x ] ' to force it to be placed in normal logic cells instead of inferring it into ram cells.  This might not be an issue with Xilinx and the pipe size might not need to be so huge.

[promach]

Reading https://www.verilogpro.com/clock-domain-crossing-part-1/ makes me ask this question:

Is conventional synchronizers circuit really enough ?

[BrianHG]

Hun?  The 2 clock domains ck_0 and ck_90/270 are the same frequency, just a different phase.  Not 1.5x, not 0.75x.  The data moves through unaltered.

And besides, I have no trouble going from 2:1 and 1:2 clock domains so long as those clocks are generated by the same PLL so long as both clocks have the same phase.