DDR3 initialization sequence issue

[promach]

You start sampling at this point (typically, by setting the CE pin of your deserializer).

I do not have CE pin on my own deserializer.

[promach]

In https://www.xilinx.com/support/documentation/ip_documentation/ultrascale_memory_ip/v1_4/pg150-ultrascale-memory-ip.pdf#page=362 , why transform to Figure 17-22 from Figure 17-21 ?

Besides, how exactly is clocK-Centering activity being performed behind the scene ?

[promach]

Just check Figure 17-23



By the way, if https://github.com/promach/DDR/blob/main/phase_detector.v is used to help with clocK-Centering calibration activity in Figure 17-20, I have a concern that it would not work for the data signal D2.

there is ambiguity on whether to move D2 to the left or to the right
The direction (left or right) is dependent on D0 and D1 (at least few other signals have to be compared with D2)

Do you have any comment about calibrating D2 in this case ?

Besides, any idea why only slave is calibrated here ?

[NorthGuy]

I've never used ultrascale.

DDR3 calibration is a long process which consists of many steps. If you use SODIMM, you need to calibrate every byte lane separately and delays are generally unpredictable. When you calibrate reading you need to write known data to the chip, otherwise there's no way to figure if your reading is Ok or not. But to write data, you need to calibrate writing first. Since you cannot read anything back because the read is not yet calibrated, you need to calibrate writes without reads, for example, by using write leveling. But write leveling will only align your phase within 360 degrees. This makes the process somewhat tricky. The process requires many steps until you fully calibrate anything.

Ultrascale has calibrated programmable input/output delays. Spartan-6 does not. So, you can read the MIG docs to figure out how the calibration process works, but you won't be able to repeat this in Spartan-6.

Xilinx's IPs do the calibration dynamically upon the start.

If you have a single DDR3 chip, you can simplify the calibration, you can assume certain relationships between signals because you know the physical layout and know how the length matching is done, etc. So, your process may be lot simpler than what Xilinx does. Also, you can calibrate once and store the calibration data.

[BrianHG]

When you calibrate reading you need to write known data to the chip, otherwise there's no way to figure if your reading is Ok or not. But to write data, you need to calibrate writing first. Since you cannot read anything back because the read is not yet calibrated, you need to calibrate writes without reads, for example, by using write leveling.

From the portions of Altera's official DDR3 ram controller which arent encrypted, they have a test ram calibration during power-up.  They do use a random number generator to generate write test data and read it back to confirm no bit errors.  To do thing properly, your calibration test sequence should implement such a strategy to confirm the data integrity of a random data pattern during power-up.

[NorthGuy]

From the portions of Altera's official DDR3 ram controller which arent encrypted, they have a test ram calibration during power-up.  They do use a random number generator to generate write test data and read it back to confirm no bit errors.  To do thing properly, your calibration test sequence should implement such a strategy to confirm the data integrity of a random data pattern during power-up.

Xilinx also does PRBS tests at the startup.

[promach]

But to write data, you need to calibrate writing first. Since you cannot read anything back because the read is not yet calibrated, you need to calibrate writes without reads, for example, by using write leveling. But write leveling will only align your phase within 360 degrees. This makes the process somewhat tricky. The process requires many steps until you fully calibrate anything.

As for "write leveling will only align your phase within 360 degrees." , write leveling could only help to align DQS strobe with CK signal.  So, it is still not enough to perform the clocK-Centering activity for those parallel write data DQ bits signals.

Besides, see also https://www.xilinx.com/support/documentation/white_papers/wp249.pdf#page=5

[promach]

Furthermore, someone told me that I shall encode the delay tap of IODELAY2 primitive as gray code (0, 1, 3, 2, 6, 7, 5, 4, C, D, F, E, A, B, 9, 8 ) .  But does it really matter whether it is gray code or not for IODELAY2  ?

[NorthGuy]

As for "write leveling will only align your phase within 360 degrees." , write leveling could only help to align DQS strobe with CK signal.  So, it is still not enough to perform the clocK-Centering activity for those parallel write data DQ bits signals.

Write leveling is just one of the (first) steps.

BrianHG assumed that DQS and CK are already aligned, and this worked Ok.

Most controllers simply write DQ with a clock which is nominally 90 degree shifted from the clock they use to write DQS, assuming that DQ and DQS are length matched and have minimum skew.

It is up to you to decide what you're going to calibrate and how.

Furthermore, someone told me that I shall encode the delay tap of IODELAY2 primitive as gray code (0, 1, 3, 2, 6, 7, 5, 4, C, D, F, E, A, B, 9, 8 ) .  But does it really matter whether it is gray code or not for IODELAY2  ?

I always thought the Xilinx's delays are glitch free.

If you delay the clock (e.g. DQS) and the clock is fast, instead of worrying about glitches, I would rather worry about smoothness of changes and do 0,1,2,3 ... rather than allowing big changes (e.g. from 4 to C).

[promach]

I always thought the Xilinx's delays are glitch free.

If you delay the clock (e.g. DQS) and the clock is fast, instead of worrying about glitches, I would rather worry about smoothness of changes and do 0,1,2,3 ... rather than allowing big changes (e.g. from 4 to C).

See Glitchless delay line using gray code multiplexer

[promach]

Most controllers simply write DQ with a clock which is nominally 90 degree shifted from the clock they use to write DQS, assuming that DQ and DQS are length matched and have minimum skew.

It is up to you to decide what you're going to calibrate and how.

@NorthGuy  You mean the following skew situation would not actually happen for write activity ?

[promach]

now, I am checking if I could eliminate either cal_master or cal_slave signal since I am only using one single whole deserializer

I suspect the same phase calibration mechanism could be done without using an extra SLAVE ISERDES

[NorthGuy]

@NorthGuy  You mean the following skew situation would not actually happen for write activity ?

This picture is from QDR II memory chapter. I never worked with QDR II, nor with ultrascale, and know very little about either. I cannot comment on this.

In DDR3, delay through DQ and DQS of the same byte lane is equalized through the length-matching of the PCB traces, so if you nominally shift DQ 90 degrees from DQS during writes, then everything should get aligned on the DDR3 side as well. There will be some skew of course. But it will be small enough and will never be as big as whole bit. If your FPGA has an ability to delay output lines, you can try to calibrate it out if you wish. Most controllers won't bother.

Think about it. If you use 600 MHz clock, the distance between bytes is roughly 800 ps. The skew between DQ lines of the same byte lane cannot be that big. I suggest you get a board with DDR3 (preferably with SODIMM to observe disparity between byte lanes, but single/dual chip are Ok too) and measure actual skews and windows.

As an example, look how this guy built his DDR3 controller on Zynq:

https://www.elphel.com/blog/2014/06/ddr3-memory-interface-on-xilinx-zynq-soc-free-software-compatible/

His Zynq was Kintex based, so he had much better hardware than Spartan-6. Therefore, his controller will not work for you. But look at his approach - it is very constructive.

[NorthGuy]

now, I am checking if I could eliminate either cal_master or cal_slave signal since I am only using one single whole deserializer

I suspect the same phase calibration mechanism could be done without using an extra SLAVE ISERDES

They use master and slave because they re-calibrate dynamically while maintaining communications at the same time. This adjusts the delays to ongoing temperature and voltage variation.

Alternatively, you can do the calibration once (either at startup or with a separate design), remember the delay values and use them.

[promach]

They use master and slave because they re-calibrate dynamically while maintaining communications at the same time. This adjusts the delays to ongoing temperature and voltage variation.

Is it possible to use only a single whole deserializer (instead of splitting into master and slave deserializers) to achieve the similar quoted purpose of dynamic phase calibration ?

[promach]

How to correctly split a single 'dq_w_oserdes' SDR signal into two ('dq_w_d0', 'dq_w_d1') SDR signals for ODDR2 primitive ?

Note: dq_w_oserdes signal is a serialized output from serializer.v

In other words, how to generate the D0 and D1 input signals for ODDR2 primitive ?

[NorthGuy]

In other words, how to generate the D0 and D1 input signals for ODDR2 primitive ?

If you use hardware OSERDES, you don't need to. Just specify DDR mode for your OSERDES.

IF you write your own - you'll need two - one for D0 and one for D1.

[promach]

IF you write your own - you'll need two - one for D0 and one for D1.

I have an SDR SERDES running at n clock rate (posedge of 303MHz), but the ODDR2 need to receive new bit (D0 or D1, depending on posedge or negedge) every 2n clock ticks (both posedge and negedge of 303MHz).

I do not understand how is having two separate OSERDES would help to solve the clock rate matching issue ?

[NorthGuy]

IF you write your own - you'll need two - one for D0 and one for D1.

I have an SDR SERDES running at n clock rate (posedge of 303MHz), but the ODDR2 need to receive new bit (D0 or D1, depending on posedge or negedge) every 2n clock ticks (both posedge and negedge of 303MHz).

I do not understand how is having two separate OSERDES would help to solve the clock rate matching issue ?

For example, look at 8:1 DDR OSERDES which takes 8 inputs D0,D1,D2,D3,D4,D5,D6,D7 and output them serially

The values supplied by D0,D2,D4,D6 are clocked out on the rising edge
The values supplied by D1,D3,D5,D7 are clocked out on the falling edge

You can then create two 4:1 SDR OSERDES modules.

One of the 2 modules will take D0,D2,D4,D6 inputs and output them serially. You route its output to the D0 pin of the ODDR.

The other will output D1,D3,D5,D7 serially. You route its output to the D1 pin of the ODDR.

But this is only if you write your own OSERDES.

The hardware OSERDES will have built-in DDR mode. Even if you put it in SDR mode, it cannot be routed to ODDR because ODDR and OSERDES are two incarnations of the same OLOGIC block.

[promach]

@NorthGuy

Thanks for you advice on using two separate OSERDES.  See https://github.com/promach/DDR/blob/main/ddr3_memory_controller.v#L777-L897 for the code modification

Now, I am back to solving the skew calibration issue. I need to study more on Write Leveling as well as MPR_Read_function.

By the way, how to move the DQS strobe to the center of write DQ bits ?

[promach]

Another separate question on bitslip:

In https://www.xilinx.com/support/documentation/user_guides/ug190.pdf#page=366 ,  how to derive the table for DDR mode inside Figure 8-10 ?

[NorthGuy]

By the way, how to move the DQS strobe to the center of write DQ bits ?

I assume you're talking about writes. The easiest way is to generate a separate clock for DQ which is shifted by 90 degree from the clock that generates DQS.

Another separate question on bitslip:

In https://www.xilinx.com/support/documentation/user_guides/ug190.pdf#page=366 ,  how to derive the table for DDR mode inside Figure 8-10 ?

As it says, when you apply BITSLIP it rotates 1 bit to the right, the next time it rotates 3 bits to the left and so on. This may be different in Spartan-6.

[promach]

As it says, when you apply BITSLIP it rotates 1 bit to the right, the next time it rotates 3 bits to the left and so on. This may be different in Spartan-6.

The rationale of using 3 bits rotation is a bit strange though ?


By the way, must I use IODELAY2 primitive in order to shift the incoming READ DQS strobe to the center of the parallel DQ bits ?

[NorthGuy]

The rationale of using 3 bits rotation is a bit strange though ?

This is probably a consequence of the construction. They use the same block to do lots of things, so it is not always logical.

By the way, must I use IODELAY2 primitive in order to shift the incoming READ DQS strobe to the center of the parallel DQ bits ?

I think so. Depending on the path the signal takes inside FPGA they will also have delays of their own, which you need to take into account.

[promach]

As for the use of IODELAY2 primitive to shift READ DQS strobe to the center of the incoming parallel DQ bits,  which signal should be used for port 'CLK'  ?  Is it the slow system clock, or the PLL-clk (minimum 303MHz) ?

I have a feeling that none of these two clocks should be used for port 'CLK' of IODELAY2 because READ DQS strobe itself is sort of already acting as clock activity.