6GSps FPGA to Memory Handling...

[Mechatrommer]

as someone said it, let me consider a theoretical idea of a spherical dead cow in a vacuum (whatever that means is)... this is just out of my imagination...

lets say i have 4X 8 bit ADC sampling at 1.5GSps each, and then all of the data are sent in staggered manner like Rigol DSO did, or just simultaneously into a FPGA to get effective 6GSps acquisition. and then the data are sent to memory RAM bank (1 bank is enough? or 4 banks is ok?). if 1 memory bank/chip per adc architecture is used (ie 4 banks total), at 1.5GSps data rate, i'm guessing for parallel interface, the clock is the same as that or twice. if using serial interface, then the clock at least 8X of that ie 12GHz, i dont see anything closer to that in digikey (2GHz is the RAM max clock in stock/list), am i delusional here? so question...

1) is there a FPGA that can handle this magnitude?
2) same as RAM? from digikey parallel interface @ > 1.5GSps, there are few candidates, but mostly no picture, not in single quantity purchase, or obsolete model.

Waiting your reply thanks.

[NiHaoMike]

6GS/s at 8 bits is 6GB/s, which is a lot but common DDR3 easily does more than that.

[Mechatrommer]

6GS/s at 8 bits is 6GB/s, which is a lot but common DDR3 easily does more than that.

so i guess 1 RAM per ADC is a wise choice. 1.5GB/s should give more lee-way to the requirement and broader option in RAM IC... i suspect a FPGA should be powerfull enough to send 32bits data to RAMs @ 1.5GHz clock rate, no? i'll search for it later, i just dont know which specification is 6GHz and how many cells and internal memory required.

[NiHaoMike]

The tricky part is going to be the PCB design. Just look at some PC motherboards to get an idea of how intricate high speed wiring is.

[asmi]

There are few issues here.

First - data acquisition interface. A lot of really high-speed ADCs have JESD204B interface, and for that you will need to have FPGA with MGTs to interface with them as regular IO pins are not fast enough. If you use Artix-7, there are one or two MGT quads (4 tx and 4 rx each, the only device with two quads is the largest Atrix-200). If the ADC of your choice has LVDS interface, this will need quite a bit of pins and quite fast SERDES. There is a couple of "traps for young players" though - MGT can only do up to 3Gbps at speed grade -1, you will need -2 to get full 6G, and the same goes for SERDES - at grade 1 they can only go up to 950 Mbps in DDR mode, while they will go up to 1250 Mbps at speed grade 2.

Second, memory. Same Artix-7 can do 800 MT/s (400 MHz x 2 transfers per clock), so you can get 800 MB/s for a byte-wide interface (x8). The widest you can do is x64 (you will need to use FBGA-676 package for that), that will give you 8 x 800 = 6400 MB/s = 6.25 GB/s. But this is the theoretical maximum, real bandwidth is going to be less than that because DDR3 needs to be refreshed every once in a while. If you want faster, you will either have to run the memory at 533 MHz (that will yield 1066 MT/s, but that is only possible for speed grade 3 devices), of use Kintex-7 devices which can run the memory at up to 1600 MT/s, which is essentially a double of what Artix can do.

[asmi]

i suspect a FPGA should be powerfull enough to send 32bits data to RAMs @ 1.5GHz clock rate, no?

No way that's happening. Nothing can move inside FPGA at that kind of rate (excluding MGTs, but they work as SERDES where their parallel interface works relatively slowly).

[BrianHG]

i suspect a FPGA should be powerfull enough to send 32bits data to RAMs @ 1.5GHz clock rate, no?

No way that's happening. Nothing can move inside FPGA at that kind of rate (excluding MGTs, but they work as SERDES where their parallel interface works relatively slowly).

800Mhz clock on DDR3 (This means 1.6 gigawords per pec), Altera Stratix V and Arria V GV can handle that and then some.  (Some of these need QDR ram to get this speed, others can do it with off the shelf DDR3/DDR4) Read here: (I would still use 64bit ram, most likely a SODIMM ram module, at this rate, you can use Stratix IV, Arria V)
https://www.altera.com/support/support-resources/external-memory.html

Use a smart ram controller with a smart crossed interleaved upper/low\er address page banking, when sampling, your ram will never need a refresh cycle.  (Done this trick in my video scan rate converter, however, I wrote my own 8 port ram controller and the smart refresh generator 'ticked' off memory pages being access, so, when rendering video continuously, the memory would almost never need a refresh cycle.  I would have to turn off all the video access for the refresh cycles needing to begin.)

If you want some good clearance with only a single 32bit port, you will have to use the Intel Arria 10, which can run DDR4 at 1.2Ghz, or 2.4 gigawords/sec, or, DDR3 at 1.066Ghz.  (You are cutting things really close with DDR3, don't forget there is overhead and you can only effectively get this speed with nice clean sequential page bursts.  Trying to address and write single bytes wont cut it.  Also, don't forget that in the FPGA, your cache buffers are running in a double or quadruple sized buss at maybe 1/4 your sample clock rate.  IE, 8 bits at 1.5Ghz are cached as 32 bits at 375Mhz.)

This is big $$$.  When Stratix 10 becomes available, it's internal cache ram can only operate at 1GHz, though you get that beautiful 2.6GHz DDR4 support.

[asmi]

800Mhz clock on DDR3 (This means 1.6 gigawords per pec), Altera Stratix V and Arria V GV can handle that and then some.  (Some of these need QDR ram to get this speed, others can do it with off the shelf DDR3/DDR4) Read here: (I would still use 64bit ram, most likely a SODIMM ram module, at this rate, you can use Stratix IV, Arria V)
https://www.altera.com/support/support-resources/external-memory.html

That is Kintex-7 territory. They can also run DDR3 at 800 MHz (1600 MT/s), but unlike greedy Antel Xilinx allows you to do that using K70/K160 devices without paying a single cent for the software or IPs. For me personally that's a big advantage. Moreover, even some cutting edge UltraScale/UltraScale+ devices are included in free WebPack edition - but here it's getting to the territory where the chip's price starting to approach a ballpark of what license would cost. Still - I'd rather buy 6 Zynq UltraScale+ MPSocSs (~$500 each) than spend that money on a license

If you want some good clearance with only a single 32bit port, you will have to use the Intel Arria 10, which can run DDR4 at 1.2Ghz, or 2.4 gigawords/sec, or, DDR3 at 1.066Ghz.

That is multi-thousand $$ affair, Xilinx UltraScale/US+ can do that too (DDR4 up to 1333 MHz IIRC). But that doesn't mean that an internal bus can run that fast. If my memory serves me, even on super hi-end FPGAs you can only run logic at like 600-700-800 MHz. That's what I meant when I said "No way to run anything inside at 1.5 GHz".

[NiHaoMike]

A long time ago, ATI (before they merged with AMD) did make GPUs with video input for connecting cameras. I think the real problem is lack of demand.

[ejeffrey]

I've got a design that does that with DACs.  8x AD9736 connected to an Arria V FPGA.  Those are 14 bit DACs operating at 1 GS/s.  They are kind of old school and use a parallel LVDS interface, so that is 120 LVDS pairs on the FPGA.  You can do the same with ADCs, and up to 1-2 GS/s there are still available with LVDS.  Faster chips typically use JESD204b, which requires an FPGA with lots of 10G+ transceivers.

We only use SRAM.  DDR memory has plenty of bandwidth, but to have both DDR memory interface *and* 120 LVDS transmitters would require a much larger FPGA.

The big gotcha with DRAM is the refresh.  You don't want the ADC to stall out because of a memory refresh.  That means you either have to have a big SRAM buffer or find a way to avoid hitting the refresh.  The way to make that work is application dependent.

[Mechatrommer]

sorry i wasnt clear enough about this, i'm so naive in this thing... i dont even know how to program in FPGA i wish there is MCU to do this kind of thing, but at GSps multibus rate, FPGA seems inevitable. i was looking at https://www.digikey.my/product-detail/en/texas-instruments/ADC081000CIYB-NOPB/ADC081000CIYB-NOPB-ND/1302378 ADC so the data interface is 16 bits LVDS (i'll need to study more on this, your advice will always be more than welcomed), so its effectively 800MSps @ 16 bits X 4 ADC.

If you want some good clearance with only a single 32bit port, you will have to use the Intel Arria 10, which can run DDR4 at 1.2Ghz, or 2.4 gigawords/sec, or, DDR3 at 1.066Ghz.

That is multi-thousand $$ affair, Xilinx UltraScale/US+ can do that too (DDR4 up to 1333 MHz IIRC).

this one of the main concern, i dont want to be left with $500 useless part that cant do the job just as good as wall decoration in frame. now i got few names to start with.

I've got a design that does that with DACs.  8x AD9736 connected to an Arria V FPGA.  Those are 14 bit DACs operating at 1 GS/s.  They are kind of old school and use a parallel LVDS interface, so that is 120 LVDS pairs on the FPGA.  You can do the same with ADCs, and up to 1-2 GS/s there are still available with LVDS.  Faster chips typically use JESD204b, which requires an FPGA with lots of 10G+ transceivers.

We only use SRAM.  DDR memory has plenty of bandwidth, but to have both DDR memory interface *and* 120 LVDS transmitters would require a much larger FPGA.

The big gotcha with DRAM is the refresh.  You don't want the ADC to stall out because of a memory refresh.  That means you either have to have a big SRAM buffer or find a way to avoid hitting the refresh.  The way to make that work is application dependent.

hey! i didnt know there is 14 bits ADC @ > 1GSps that is half the price of the 8 bits ADC081000 ADC. thanks! i maybe consider AD9736 in the design plan (that is lingering in the mind) but BGA footprint is no fun for hobbiest one-off project.

to be more specific, there are 4 "paths" for the design (maybe 5 or 6 paths @ 1GSps depending on part used), lets just take one, the other 3-5 will just duplicates. an ADC (8 - 14 bits @ 500 - 800MHz data clock) -> FPGA -> RAM. the total RAM is about 50 - 100MBytes. so its 12.5 - 25MB per RAM (4 paths) or 8 - 16MB per RAM (6 paths). all of this 100MB must be filled at effective rate of 6GSps (total 4x ADC), and then later there will be blind time (no data capture) where another MCU, or the same FPGA will process this data, FFT to be exact, before sending it to PC (USB) or screen for human viewing. anyway i think if i go RAM -> CPU path for FFT processing, this is the easiest part (so i also need a RAM that can be clocked a lot slower to interfacce to MCU), but if i go FFT in FPGA route, this may open another can of worm for a discussion, short FFT up to 1024 pts is common, but MBytes length of FFT is another thing... https://www.ll.mit.edu/HPEC/agendas/proc04/powerpoints/Posters/Poster%20B/dillon_posters.ppt let alone staggered clock to get 6GSps from 4 x 1.5GSps ADC parts.

so the total FPGA pins required is 4 (ADC) X 16 x 2 (LVDS) = 128 pins, thats for input only (ADC), i guess for RAM requires the same pins amount, so total 256 pins, excluding other pins for comm/command/synch etc... man!

this discussion has lead me to few app notes... google "lvds to fpga" (for my reference later)...
https://www.altera.com/en_US/pdfs/literature/an/an479.pdf
https://www.xilinx.com/support/documentation/application_notes/xapp1071_V6_ADC_DAC_LVDS.pdf
but not quite ADC->FPGA->RAM thing. anyway this is good when we have somebodies helping shoulder to shoulder, thanks guys.

[ali_asadzadeh]

Following

[C]

Instead of 4 ADC's into one fpga, you might think of 4 ADC's synced into 4 fpga's.
With good design this could expand by ADC.

Worked with a CAD system back in 80's that had each thing as a separate bit plane. Made actual processing easy. A simple logic board then turned many planes into video.

Todays 1.5GSps could become tomorrows 3GSps with a 3GSps input to two 1.5GSps output block.

[NorthGuy]

FPGA is not MCU - you decide on the bus width and you can go as wide as you want. Thus, this is mostly a question of available pins.

On the ADC side it'll be expensive. For example, 2 of ADC08D1520 or one EV8AQ160 - approaching $1K. That'll take around 100 pins.

On the memory side, this is a matter of bus width. 1.5GS/s x 8 x 4 is 48 Gb/s. If you use a 64-bit DDR3 (standard DIMM), then you need 48/64 = 667 Mb/s. If you don't get much overhead then 800 Mb/s (400 MHz clock) will be enough. That's around 120 pins.

So, you'll need 100 + 120 = 220 pins - quite feasible in one FPGA, such as Spatan-7, or Artix-7 if you decide that 484-pin package is too tough. FPGA will cost much less than what you pay for ADCs.

[ejeffrey]

hey! i didnt know there is 14 bits ADC @ > 1GSps that is half the price of the 8 bits ADC081000 ADC. thanks! i maybe consider AD9736 in the design plan (that is lingering in the mind) but BGA footprint is no fun for hobbiest one-off project.

DACs are generally cheaper and higher resolution than ADCs.

The ADC081000 is not recommended for new designs.  We are also using it in one of our designs (only 2 channel) When I did a survey, the best replacement I found was the ADC8D1020.  Unlike the ADC081000, it runs the LVDS pairs at 1 Gb/s instead of 500 MBit/s.  This cuts your needed LVDS pairs in half, and also makes synchronizing easier.

[Mechatrommer]

hey! i didnt know there is 14 bits ADC @ > 1GSps that is half the price of the 8 bits ADC081000 ADC. thanks! i maybe consider AD9736 in the design plan (that is lingering in the mind) but BGA footprint is no fun for hobbiest one-off project.

DACs are generally cheaper and higher resolution than ADCs.

epic fail i tought its a ADC...

The ADC081000 is not recommended for new designs.  We are also using it in one of our designs (only 2 channel) When I did a survey, the best replacement I found was the ADC8D1020.  Unlike the ADC081000, it runs the LVDS pairs at 1 Gb/s instead of 500 MBit/s.  This cuts your needed LVDS pairs in half, and also makes synchronizing easier.

but ADC8D1020 is not available in digikey nor ebay... sad.

On the ADC side it'll be expensive. For example, 2 of ADC08D1520 or one EV8AQ160 - approaching $1K. That'll take around 100 pins.

i thought FPGA cost (SW license and the chip) will be few magnitude more expensive. on the adc side, thats why we go Rigol's style. 4 X ADC081000 (6GSps staggered) is the cost of 1 X EV8AQ160 (3GSps), i cant find ADC08D1520 price in digikey...

[nctnico]

It depends on what the ADC interface is but if you have a wide memory bus (like 128 or 256 bit) then you can run the memory at much lower speeds. You'll need to implement your own DDR controller though.

[NorthGuy]

i thought FPGA cost (SW license and the chip) will be few magnitude more expensive. on the adc side, thats why we go Rigol's style. 4 X ADC081000 (6GSps staggered) is the cost of 1 X EV8AQ160 (3GSps), i cant find ADC08D1520 price in digikey...

https://www.digikey.com/product-detail/en/texas-instruments/ADC08D1520CIYB-NOPB/296-41497-ND/1870663

Also:

https://www.digikey.com/product-detail/en/texas-instruments/ADC08D1020CIYB-NOPB/ADC08D1020CIYB-NOPB-ND/1762580

Xilinx's Vivado is free for smaller chips. DDR3 IP is free.

Entry level FPGAs which may fit your needs:

https://www.digikey.com/product-detail/en/xilinx-inc/XC7S50-2FGGA484I/122-2046-ND/7389946

https://www.digikey.com/product-detail/en/xilinx-inc/XC7A15T-2FGG484I/122-1931-ND/5248118

If you want to go ballistic (but still free Vivado):

https://www.digikey.com/product-detail/en/xilinx-inc/XC7A200T-3FFG1156E/122-1858-ND/3925786

[asmi]

If you want to go ballistic (but still free Vivado):

https://www.digikey.com/product-detail/en/xilinx-inc/XC7A200T-3FFG1156E/122-1858-ND/3925786

There is no point going with this device, as Kintex-160 is about the same price, while it will give you double memory bandwidth and double transceivers speed.

[BrianHG]

Cyclone V supports binding two 32-bit, DDR3/3L 800MT/s controllers to form a single 64-bit, 800MT/s controller, which gives you a theoretical BW of 6.4GB/s.
You need to keep the flow of data continuous, otherwise random addressing (CL time) will eat away your actual BW quickly.

Xilinx entry level Artix supports 1066MT/s, so you have a bit more margin.

I would be worried about both being so close to the nose and the core clock speed for the internal dual port memory cache ram.
ArriaV would bring him from CycloneV 800MT/s to ArriaV 1336MT/s in an approximate 350$ IC, and internal core memory speeds from CycloneV 300-420Mhz to ArriaV 400-500Mhz.  The down side is you need to buy Quartus to work with the ArriaV where it is free for the simpler variants of CycloneV.

I would still go 64 bit ram on the ArriaV and give a wide margin for play.  You never know...  Slightly off PCBs, bad impedance controls, shady DDR3 memory, poor assembly, all which might lead you to lowering your DDR3 clock rate.  With memory running at max 10GB/sec, you have a clear 1.5x headroom which will allow row precharge & refresh cycles and you can safely slow-down your ram clock to around 1Ghz from 1.336Ghz top speed.  800Mhz is just too close and would fail without ultra firmware tweaking.

[NorthGuy]

If you want to go ballistic (but still free Vivado):

https://www.digikey.com/product-detail/en/xilinx-inc/XC7A200T-3FFG1156E/122-1858-ND/3925786

There is no point going with this device, as Kintex-160 is about the same price, while it will give you double memory bandwidth and double transceivers speed.

Not really. Here's a comparably priced Kintex:

https://www.digikey.ca/product-detail/en/xilinx-inc/XC7K160T-1FBG676C/122-1961-ND/3911247

It has less pins, less logic. Since it's grade -1, while Artix is -3, the speed is roughly the same. DDR3 is rated at 800MT/s for Kintex while the rating for Artix is 1066MT/s, so Artix is actually a little bit faster.

[asmi]

Not really. Here's a comparably priced Kintex:

https://www.digikey.ca/product-detail/en/xilinx-inc/XC7K160T-1FBG676C/122-1961-ND/3911247

It has less pins, less logic. Since it's grade -1, while Artix is -3, the speed is roughly the same. DDR3 is rated at 800MT/s for Kintex while the rating for Artix is 1066MT/s, so Artix is actually a little bit faster.

This one is even cheaper: https://www.digikey.com/products/en?keywords=XC7K160T-1FFG676C but it supports 1600 MT/s for DDR3. It's got a bit less logic and 100 less IOs, but 400 IOs is still more than enough for the project at hand, while faster memory and MGTs are definitely useful in case you want to interface via JESD204B. Also there is a lot of headroom for growth in case you will need more logic (but you'll have to buy a license), while the Atrix-200 is a dead end.

[BrianHG]

Not really. Here's a comparably priced Kintex:

https://www.digikey.ca/product-detail/en/xilinx-inc/XC7K160T-1FBG676C/122-1961-ND/3911247

It has less pins, less logic. Since it's grade -1, while Artix is -3, the speed is roughly the same. DDR3 is rated at 800MT/s for Kintex while the rating for Artix is 1066MT/s, so Artix is actually a little bit faster.

This one is even cheaper: https://www.digikey.com/products/en?keywords=XC7K160T-1FFG676C but it supports 1600 MT/s for DDR3. It's got a bit less logic and 100 less IOs, but 400 IOs is still more than enough for the project at hand, while faster memory and MGTs are definitely useful in case you want to interface via JESD204B. Also there is a lot of headroom for growth in case you will need more logic (but you'll have to buy a license), while the Atrix-200 is a dead end.

+1 for the 1600MT/sec.  Like I said, double your bus size on the ram and get that 2x clearance, which realistically will only be around 1.5x after overhead.  1000MT/sec as a limit is just cutting things way too close.  You have near 0 wiggle room.

My choice would be go for the Arria 10GX '10AX022C4U19E3SG' for 410$, https://www.digikey.com/products/en?keywords=10AX022C4U19E3SG
You get 13 megabit onchip cache at 730 MHz.
191 hardware floating point multipliers/adders for FFT with a peak 172 GFLOPS.
You should be able to feed the 1.6Ghz from your ADC directly into this Arria as it has way more than 32 1.6GHz LVDS channels, though, you will need to read through using a dedicated IO bank and dedicated PLL & differential clock channel as well, but the IOs and internal deserializes are designed for that sort of thing.
2400MT/sec DDR4 memory interface, run it at 64 bit.  Then you have enough ram speed to sample, process, and run onboard software all from 1 ram bank.  Or, even add a second smaller 32bit ram controller section.

NOTE:  Never buy a singe Altera/Xilinx IC before building the timing critical guts of your project in their dev suites & doing a full timing simulation.  Otherwise, you may be throwing out money on something which wont be fast enough.

You can learn how to do things on Altera's free web version of Quartus with a slower FPGA.  Altera also allows 1 free month with the full version once you are ready to simulate a full speed Arria 10 version, but, to program the Arria 10, you will need a license.

Others here can offer you better advice on using Xilinx, but, I still say do a full timing simulation with real-world simulated inputs, ram controller and additional processing.  Otherwise, you might design and build and buy something which wont be fast enough.

[NorthGuy]

it supports 1600 MT/s for DDR3.

1600 MT/s DDR3 is a plus because it lets you use 32-bit wide memory, while with 800 MT/s you'd need at least 64-bit wide. This saves 40 pins and some board space. This puts more strain on board design in terms of length matching and SI.

If you decide on 1600 MT/s, then Kintex will be your starting point. I would rather start from small Spartans and Artixes, which are only $50 (or less) and might be quite sufficient for the task.

[asmi]

1600 MT/s DDR3 is a plus because it lets you use 32-bit wide memory, while with 800 MT/s you'd need at least 64-bit wide. This saves 40 pins and some board space. This puts more strain on board design in terms of length matching and SI.

I'd use 64bit SODIMM module as the socket tends to make layout easier. And at 1600 x 8 = 12.5 GB/s of bandwidth there is a huge margin.

If you decide on 1600 MT/s, then Kintex will be your starting point. I would rather start from small Spartans and Artixes, which are only $50 (or less) and might be quite sufficient for the task.

I don't think they will be sufficient - certainly not $50 ones. Also in some ways using Kintex is easier because they don't require small decoupling caps under BGA as they are already on a package. So you only need to fit a small amount of bulk caps (4.7 uF and up). This essentially frees up the bottom layer for additional routing space.