FPGA VGA Controller for 8-bit computer

[RoGeorge]

Wow you're designing all this without a scope?

There's not much an oscilloscope can tell with FPGAs.

The inside of the FPGA can not be probed, and routing internal signals to outside pins just to probe something is not an option.  That would change too much the design, might make the timing constraints to fail, etc., not to say big designs can take hours to build a new bitstream.

The workhorse in digital design is the simulator.  Once the simulation works, and the timing closure is satisfied, it should work.  If there is still something to debug, then ChipScope (Xilinx) / SignalTap (Altera) would the tool to capture and watch internal signals.  Sometimes a JTAG might help, too.  In rare situations, at low speed, a logic analyzer might also help.

Only in very rare situations one may want to probe FPGA signals with an oscilloscope, usually when hunting for signal integrity issues, for example to see how open is the eye on a high speed SERDES, or something alike.

[NorthGuy]

On single ended clocks you can detect DC bias with DMM. Say, with 3.3V signal, the DMM will read 1.6V if the clock is present.

Not so with differential signals terminated at VDD/2

[BrianHG]

The DDR3 is coming.... Having a scope would have saved me almost a week as the CLK out to the ram wasn't oscillating due to unknown BS as it simulated fine both at logic and gate level properly, sound design, but something about the MAX10 as the chosen IOs just generated dead output.  It's a pain avoiding a bunch of Altera's eccentricities as I avoid their built in DDRx PHY and do everything in a way which can be supported throughout all Altera FPGAs as well as be ported to other FPGA vendors.

Wow you're designing all this without a scope?

Damn clean coding and proper simulation with the DDR3 model...

Yes, Altera offers 'SignalTap' logic analyzer through the J-Tag just like there is one in Lattice Diamond and Xilinx, however, this stupid bug was me using the CLK output pin as a DDR output,  1/2 clock signal high and the other low to reconstruct a parallel clock.  This is fine and works with most older Altera FPGAs like CycloneIV, but, the signal gets muted out for some reason on the MAX10s chosen IOs for the DDR3.

The Max10/CycloneV on the CK and DQS lines has a nice feature which offers 128x15ps taps on those IOs with a feedback so you may tune your sample times specifically for DDR3/4 interfaces.  This also means less taps on the PLL.  But, If I use the function, my code would not work older Cyclones & third party vendors FPGAs.  These 2 chips also offer a dedicated 2 way FIFOs on the DQ lines merges with DQS and CK for clocking which I am not using.  They can all be easily accessed and pre-packaged in 1 nice module by the free DDR2/3/4 PHY available with Quartus, but if I designed using it, where would the fun be in doing it all manually in a way which is cross-vendor compatible?

Right now, I'm phase-stepping on the PLL to create my required 3 clocks.  This is not as fine as 15ps steps, but right now I do get a valid data window of around 10 steps out of 64.  This should clean up as I'm now on the SDC file.  Yes, I'm getting good data now without any timing constraints other than 'multicycle path' for signals crossing of clock domain boundaries.

I've generated 3.5k line of code, all documented and explained with comments at almost every line...

[BrianHG]

  MAX 10 doesn't support the old altddio.  It does for input but not output.  No warnings, no compiler complaints.  It simulates properly as if the chip has the logic, but it totally ignores the output.

I have to use the ALTERA_GPIO_LITE which is specific to the newer MAX10 and above...

[NorthGuy]

The Max10/CycloneV on the CK and DQS lines has a nice feature which offers 128x15ps taps on those IOs with a feedback so you may tune your sample times specifically for DDR3/4 interfaces.  This also means less taps on the PLL.  But, If I use the function, my code would not work older Cyclones & third party vendors FPGAs.  These 2 chips also offer a dedicated 2 way FIFOs on the DQ lines merges with DQS and CK for clocking which I am not using.  They can all be easily accessed and pre-packaged in 1 nice module by the free DDR2/3/4 PHY available with Quartus, but if I designed using it, where would the fun be in doing it all manually in a way which is cross-vendor compatible?

Xilinx 7-series FPGAs also have delay lines and designated shallow FIFOs for IO.

[asmi]

Xilinx 7-series FPGAs also have delay lines and designated shallow FIFOs for IO.

Also undocumented phasers which you will have to use for write leveling because there are no output delay blocks in HR banks.
So, yeah - I stand by my doubts about vendor-neutral PHYs...

[BrianHG]

The Max10/CycloneV on the CK and DQS lines has a nice feature which offers 128x15ps taps on those IOs with a feedback so you may tune your sample times specifically for DDR3/4 interfaces.  This also means less taps on the PLL.  But, If I use the function, my code would not work older Cyclones & third party vendors FPGAs.  These 2 chips also offer a dedicated 2 way FIFOs on the DQ lines merges with DQS and CK for clocking which I am not using.  They can all be easily accessed and pre-packaged in 1 nice module by the free DDR2/3/4 PHY available with Quartus, but if I designed using it, where would the fun be in doing it all manually in a way which is cross-vendor compatible?

Xilinx 7-series FPGAs also have delay lines and designated shallow FIFOs for IO.

Those are available in every FPGA's basic DDR phy toolkit.  It is the only way to ensure getting valid data in and out  from one phase to the next.  Note that there is a module in my code which gets swapped depending on selected vendor and it doe use each vendor's DDR IO buffers and dedicated first IO cells at the pins.  Otherwise this could never work.

The DDR3 provides test alignment Read and Write timing signals which can be used to align your time-of-flight feedback clocks.  Since I've designed an FPGA centric controller designed for 1 or 2 ram chips (4 of them in a row with the CK pins received inbetween them) with the DDR_CK which must be wired between them if you are using 2, I only do the read-back tuning/alignment, expecting all the signals to arrive at the ram based on the SDC constraints while the memory itself is an unknown device with unknown PCB characteristics which needs to be tuned for on powerup.  You wont be able to use DDR3 modules.

[asmi]

Since I've designed an FPGA centric controller designed for 1 or 2 ram chips (4 of them in a row with the CK pins received inbetween them) with the DDR_CK which must be wired between them if you are using 2, I only do the read-back tuning/alignment, expecting all the signals to arrive at the ram based on the SDC constraints while the memory itself is an unknown device with unknown PCB characteristics which needs to be tuned for on powerup.  You wont be able to use DDR3 modules.

This is a BIG limitation. Most DDR3 designs I've seen (and all designs I've done myself) use flyby routing for address/control lines because it's MUCH simpler from routing standpoint. So you can expect that address/control signals reach each module at the same time, but that time will be different for different modules.

[NorthGuy]

There are DDR3 chips with multiple dies, where the dies are connected within the chip using the fly-by topology. For example, Micron's MT41K512M16VRN.

[BrianHG]

There are DDR3 chips with multiple dies, where the dies are connected within the chip using the fly-by topology. For example, Micron's MT41K512M16VRN.

Shouldn't be a problem as each die is 8 bit and they have separate DQS for each die.  My controller would only have trouble for widely spaced DDR3s with clocks fed from one side to the other, and when you run in dual channel mode where multiple ICs share the same DQ & DQS as I am currently sampling the DQS with the read data as a reset & latch enable/data valid signal.  This would change if I switched to using the DQS as a read clock, but, this will force much deeper use of each vendor's memory DQ/DQS PHYs.  Not impossible, but a lot more reading of the data sheets and potentially some setup control code.

[NorthGuy]

Shouldn't be a problem as each die is 8 bit and they have separate DQS for each die.  My controller would only have trouble for widely spaced DDR3s with clocks fed from one side to the other, and when you run in dual channel mode where multiple ICs share the same DQ & DQS as I am currently sampling the DQS with the read data as a reset & latch enable/data valid signal.  This would change if I switched to using the DQS as a read clock, but, this will force much deeper use of each vendor's memory DQ/DQS PHYs.  Not impossible, but a lot more reading of the data sheets and potentially some setup control code.

The reason they have DQS is that the phase of DQ may wander over time relative to the clock. If you sample with a separate shifted clock instead of sampling with DQS, you're making the window much smaller. Xilinx did a similar thing. They have a hardware PHASER elements which creates a free-running phase-shifted clock to sample DQ. To make it work they must synchronize their clock with DQS every us or so. As a result, their controller cannot write continuously for long because synchronizing reads are needed.

Regardless of how you sample, there's no reason why you can't handle multiple chips with any configuration. There are two possible way of connecting multiple chips:

- shared CLK and control, but non-shared DQ/DQS, which is usually fly-by. In this case everything is the same except the clock/DQS phase relationship is different for every byte. Whatever you do in your controller for one byte can be done for multiple bytes as well, only every byte will require its own (set of) clocks.

- shared DQ/DQS, often called multi-rank. In this case, each chip will have its own CS line (perhaps ODT as well), so by selecting different CS you just talk to one chip or another. This certainly can be done no matter how you sample.

[BrianHG]

Shouldn't be a problem as each die is 8 bit and they have separate DQS for each die.  My controller would only have trouble for widely spaced DDR3s with clocks fed from one side to the other, and when you run in dual channel mode where multiple ICs share the same DQ & DQS as I am currently sampling the DQS with the read data as a reset & latch enable/data valid signal.  This would change if I switched to using the DQS as a read clock, but, this will force much deeper use of each vendor's memory DQ/DQS PHYs.  Not impossible, but a lot more reading of the data sheets and potentially some setup control code.

The reason they have DQS is that the phase of DQ may wander over time relative to the clock. If you sample with a separate shifted clock instead of sampling with DQS, you're making the window much smaller. Xilinx did a similar thing. They have a hardware PHASER elements which creates a free-running phase-shifted clock to sample DQ. To make it work they must synchronize their clock with DQS every us or so. As a result, their controller cannot write continuously for long because synchronizing reads are needed.

Regardless of how you sample, there's no reason why you can't handle multiple chips with any configuration. There are two possible way of connecting multiple chips:

- shared CLK and control, but non-shared DQ/DQS, which is usually fly-by. In this case everything is the same except the clock/DQS phase relationship is different for every byte. Whatever you do in your controller for one byte can be done for multiple bytes as well, only every byte will require its own (set of) clocks.

- shared DQ/DQS, often called multi-rank. In this case, each chip will have its own CS line (perhaps ODT as well), so by selecting different CS you just talk to one chip or another. This certainly can be done no matter how you sample.

Yes, the window range of tDQSQ (DQS, DQS# to DQ skew, per access) is smaller (better) than tDQSCK (DQS, DQS# rising to/from rising CK, CK#).  I guess I am not having trouble here since the DECA board has 1600MHz ram and these figures shrink down to +/-0.23ns / 0.1ns where are the slower 800MHz ram is twice as bad.  The center and hold of the read data is a sweet huge 2.54ns.  It cannot drift outside this window with reference to the CK, otherwise the ram chip has failed.

Damn Altera and their incomplete TBD specs.  Well, the clock IO jitter on Altera's DDRIOs is in the 200ps range while the setup and hold requirement on data input is within the 1ns range (Older cyclone specs).  Since I am phase clocking using the true-pll in the FPGA fabric, I have a ~ +/-500ps safety window and my phase tuning steps with a wide-band PLL is 195.3125ps when operating at 320MHz.  This jives with my current tests having a ~7 pll tuning steps with clean data.  This is advanced info and I am seeking about incorporating an async reset into my read FIFO & control pointers which would allow read data clocking right from the DQS inputs.  This would eliminate the tDQSCK +/-0.23ns improving that read window opening by another 460ps and eliminating the PLL read RDQ clock and it's power-up tuning logic.  (May allow overclocking of a soft-ram controller for Altera reaching or passing the maximum 430/860MHz serial DDR input specs so long as the ram will accept a slightly mushy (wouldn't have been the case if the DECA board use the dedicated PLL output to clock the ram instead of the dedicated DDR_CK output which has a a lower FMAX spec, but has fine skew tuning, larger drive current) CK/CK# and the fitter can time the DDR3 address & command lines which aren't part of the DDR DQ pins section.)

[BrianHG]

Update: Had to take a week break.  I'm on the SDC file and  either I can make the writes perfect, or the reads perfect...  But not both.  Gotta read up on the multicycle function and false_path.  Once this is setup, we will have a working ram controller...

[NorthGuy]

Update: Had to take a week break.  I'm on the SDC file and  either I can make the writes perfect, or the reads perfect...  But not both.  Gotta read up on the multicycle function and false_path.  Once this is setup, we will have a working ram controller...

You should use separate clocks for reads and writes.

When you read, the phase shift of DQ to your clock is determined by the FPGA -> CK/CK# -> DDR3 -> DQ/DQS -> FPGA path.

When you write, you need to synchronize DQ/DQS with the clock as it is seen inside the DDR3 chip, so the phase shift is determined by the difference between FPGA -> DQ/DQS -> DDR3  and FPGA -> CK/CK# -> DDR3 paths.

These phase shifts cannot be the same.

Not to mention that both of these clocks will be phase shifted relative to the clock used for address/control lines which is source-synchronous to CK/CK#.

[BrianHG]

Update: Had to take a week break.  I'm on the SDC file and  either I can make the writes perfect, or the reads perfect...  But not both.  Gotta read up on the multicycle function and false_path.  Once this is setup, we will have a working ram controller...

You should use separate clocks for reads and writes.

When you read, the phase shift of DQ to your clock is determined by the FPGA -> CK/CK# -> DDR3 -> DQ/DQS -> FPGA path.

When you write, you need to synchronize DQ/DQS with the clock as it is seen inside the DDR3 chip, so the phase shift is determined by the difference between FPGA -> DQ/DQS -> DDR3  and FPGA -> CK/CK# -> DDR3 paths.

These phase shifts cannot be the same.

Not to mention that both of these clocks will be phase shifted relative to the clock used for address/control lines which is source-synchronous to CK/CK#.

It's all configurable in the code.  The code was designed to adapt to other third party vendor's DDR IO blocks no matter how they jumble up the read/write order, endian, bus width, delay, clock direction, use or non use DQS as either clock or enable for reads and adapt to manual or automatic enable & disable paths, be it in a slave or master mode, both ways for read or write.  Everything I've done in the code is fully documented and explained so anyone can edit and make changes with relative ease.

[BrianHG]

Here is the DDR3 working perfectly (This is 100% true hardware, not a simulation...):



I've been troubled with this 1 weird read bug where is I used DQS[0]n/p from the 16 bit ram chip to align the reads, 1 out of every 1 million reads or so, the 128bit (8 word burst) read would be shifted off by 1 byte.  This is not possible with a 16 bit ram, it should have been shifted off by 2 bytes if it were an alignment error.

Reading using only the second DQS[1]n/p, the reads are perfect and the writes have always been perfect where I generate the both DQS[1:0]n/p outs.

I'm hunting down the bug and believe that it looks like something in my .SDC timing file is being ignored or misinterpreted as the problem appears timing in nature exclusive to 1 single IO pin.

Once this final bug is cleared, the multiport interface bus should take a fraction of time to develop as all my issues have related to Quartus and the newer DDR IOBuf specific to MAX 10 FPGAs.  At that point I'll start a new thread dedicated to the DDR3 controller.  There should be no trouble getting this design to work on Cyclone V and IV.  In fact, Cyclone IV may clock up to 400MHz.  I cannot say if it will work on Cyclone III.

[asmi]

Is this running on hardware, or a sim? If it's the latter, it might be just a quirk of simulation.

[BrianHG]

100% hardware.  No sim.

The glitch can not be verified in sim since the sim has been error free since day 1, like 3 weeks ago...

What you see is a utility which allows me to address read and write to the DDR3 through the RS232 port in and hex view in real-time.  I tied to a USB FT232 rs232 to LVTTL converter onto 2 pins of the DECA board's P6 header.

I will be creating a serious real demo which will use all the ram and the HDMI out on the DECA board.

Quartus is stingy with the .SDC as I used to use their old timing utility 20 years ago and I am learning the new language/specifications as we speak.  There is no proof anywhere that what I set is actually doing anything other than the final timing report saying it's a pass or fail.  The older CycloneIV allowed full timing accurate gate level simulations where I could see a true difference.  Not so for the MAX 10, you need to trust that what you have done in the SDC represents the truth.

[NorthGuy]

Here is the DDR3 working perfectly (This is 100% true hardware, not a simulation...):

Very impressive.

[nockieboy]

Here is the DDR3 working perfectly (This is 100% true hardware, not a simulation...):

Awesome work, BrianHG. 

I have been consistently shocked and awed throughout this DDR section of the thread about the complexity of getting DDR RAM to work with the FPGA.  All of this stuff is well and truly so far above my head it's leaving contrails.  It also makes me even more impressed at the technology (and effort gone into making that tech) sitting in the black and orange box under my desk that I'm using to write this post with right now, running billions of fault-free memory accesses into DDR RAM on a system I threw together myself.  I'm looking to add DDR3 to an FPGA approximately 1cm away at worst.  The effort gone into designing a motherboard that can work with slot-in sticks of RAM in a variety of configurations in the gigahertz range is impressive, to say the least.

[BrianHG]

Is this running on hardware, or a sim? If it's the latter, it might be just a quirk of simulation.

Like I said, hardware.
Glitch already found & fixed.
It's now compiling and operating with multiple settings in a really robust manner.  No tuning at power-up even needed.  The latest better setup .sdc set to the DDR3 datasheet generates a functioning design where tuning the PLL read phase is already dead center set to the powerup 0 degrees, with an equal +/-4 tune-able steps to the left or right before there is any data corruption.  Same for the powerup write DQ clock phase of 90 degrees, though, that one has a ~ +/- 8 tuning steps before data is corrupted.

[asmi]

I have been consistently shocked and awed throughout this DDR section of the thread about the complexity of getting DDR RAM to work with the FPGA.  All of this stuff is well and truly so far above my head it's leaving contrails.  It also makes me even more impressed at the technology (and effort gone into making that tech) sitting in the black and orange box under my desk that I'm using to write this post with right now, running billions of fault-free memory accesses into DDR RAM on a system I threw together myself.  I'm looking to add DDR3 to an FPGA approximately 1cm away at worst.  The effort gone into designing a motherboard that can work with slot-in sticks of RAM in a variety of configurations in the gigahertz range is impressive, to say the least.

This was also a somewhat unexpected for me, that once I got into FPGA board designs, I suddenly started understanding what exactly is going on a PC motherboard, or on video card. Now if you look at video card PCB, for example, you will see immediately - these are power supplies for the Vcore (in FPGA world this rail is typically called Vccint), these are for the memory, and here these serpentine tracks are length matched for the memory, etc. Same goes for the motherboard.

Also most motherboards are 6 layer ones, and I've done enough 6 layer boards that I can technically route a motherboard myself if I so desire. Also what's cool is that there are open source motherboards out there so that you can actually see full schematics and PCB and study all that stuff.

So all of what felt like a magic that made PC work, now suddenly makes perfect sense, and there are good technical reasons why certain things are done the way they are done. Now you can also understand why PC motherboards with a single DIMM slot per CPU channel tend to reach higher stable memory clocks (no stubs, shorter traces).

[NorthGuy]

It's now compiling and operating with multiple settings in a really robust manner.  No tuning at power-up even needed.  The latest better setup .sdc set to the DDR3 datasheet generates a functioning design where tuning the PLL read phase is already dead center set to the powerup 0 degrees, with an equal +/-4 tune-able steps to the left or right before there is any data corruption.  Same for the powerup write DQ clock phase of 90 degrees, though, that one has a ~ +/- 8 tuning steps before data is corrupted.

What's your clock frequency and how many degrees is your tunable step?

[BrianHG]

It's now compiling and operating with multiple settings in a really robust manner.  No tuning at power-up even needed.  The latest better setup .sdc set to the DDR3 datasheet generates a functioning design where tuning the PLL read phase is already dead center set to the powerup 0 degrees, with an equal +/-4 tune-able steps to the left or right before there is any data corruption.  Same for the powerup write DQ clock phase of 90 degrees, though, that one has a ~ +/- 8 tuning steps before data is corrupted.

What's your clock frequency and how many degrees is your tunable step?

I already answered your question, see here:
https://www.eevblog.com/forum/fpga/fpga-vga-controller-for-8-bit-computer/msg3527398/#msg3527398

[NorthGuy]

What's your clock frequency and how many degrees is your tunable step?

I already answered your question, see here:
https://www.eevblog.com/forum/fpga/fpga-vga-controller-for-8-bit-computer/msg3527398/#msg3527398

I probably misunderstood something. As I understood, you're clocking DQS writes with the same clock as CK/#CK, and you clock DQ writes with the other clock which is nominally 90 degrees shifted, but can tolerate +/- 8 steps. If steps are 200 ps, this equals 1.6 ns, that's roughly 180 degrees with 320 MHz clock (0 to 180 relative to DQS). If you shifted by 9 steps, you would be in another bit's territory. Looks like you don't have any jitter zone between bits - close to 100% window. Is that really so?