Author Topic: 6GSps FPGA to Memory Handling...  (Read 12262 times)

0 Members and 1 Guest are viewing this topic.

Offline NorthGuy

  • Super Contributor
  • ***
  • Posts: 3137
  • Country: ca
Re: 6GSps FPGA to Memory Handling...
« Reply #25 on: April 23, 2018, 01:58:20 pm »
I don't think they will be sufficient - certainly not $50 ones.

Why not? Acquiring 4 ADC channels to DDR3 memory doesn't require a lot of logic. The limiting factor is the number of pins. What Artix have you used on your last board? What was resource utilization for your MIG?

Of course, if there's some sort of data processing besides acquisition, this should be accounted for.

 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14297
  • Country: fr
Re: 6GSps FPGA to Memory Handling...
« Reply #26 on: April 23, 2018, 02:10:07 pm »
The trick that has been widely in use in digital oscilloscopes is to run many memory chips in parallel so as to increase throughput without needing insanely high-speed memory.
For ADCs, the trick is to have several ADCs that sample data in an interleaved way, so that each one can sample at lower effective sample rate.

We already had digital scopes with 8 or 16 gsa/s almost 20 years ago (see the DDA series of Lecroy scopes for instance) and this is how that was done.
 

Online MechatrommerTopic starter

  • Super Contributor
  • ***
  • Posts: 11535
  • Country: my
  • reassessing directives...
Re: 6GSps FPGA to Memory Handling...
« Reply #27 on: April 23, 2018, 02:53:40 pm »
I don't think they will be sufficient - certainly not $50 ones.
Why not? Acquiring 4 ADC channels to DDR3 memory doesn't require a lot of logic. The limiting factor is the number of pins. What Artix have you used on your last board? What was resource utilization for your MIG?
Of course, if there's some sort of data processing besides acquisition, this should be accounted for.
for now, the paid $4000 quartus SW is not justifiable for a one-off that probably never see the light. so using an FPGA that doesnt require software license is much preferable. earlier i said the FPGA probably will perform FFT as well, but if this means requiring larger FPGA with multi thousand dollar license, then i think it should be beneficial in term of cost to off load the FFT to MCU processing, Arm V7 1GHz quad core or the latest 64bits Arm A53 can be hard at cheap, or maybe even adding 2nd FPGA specialized in FFT alone, this... mmmv (my milage may vary). now i'm not sure on licensing scheme of other brand like the mentioned Artix KIntex etc, but if the cost of the license is near 4 digits cost then i think they have to be discarded from the list, for now while the money is still peanut small...

...have several ADCs that sample data in an interleaved way, so that each one can sample at lower effective sample rate.
yes i should have said interleaved, instead of "staggered" :palm:

We already had digital scopes with 8 or 16 gsa/s almost 20 years ago (see the DDA series of Lecroy scopes for instance) and this is how that was done.
exactly except they are proprietary, and the components cost back then i believe is unachieveably expensive by hobbiests during the time...

today, what is unachieveable is the sw license cost :palm: people just know how to get money. its understandable for FPGA development tools because they have to cover R&D and production factory, but look at things like EM Solver, the cost is just simply nuts... offtopic.
« Last Edit: April 23, 2018, 02:59:07 pm by Mechatrommer »
Nature: Evolution and the Illusion of Randomness (Stephen L. Talbott): Its now indisputable that... organisms “expertise” contextualizes its genome, and its nonsense to say that these powers are under the control of the genome being contextualized - Barbara McClintock
 

Offline asmi

  • Super Contributor
  • ***
  • Posts: 2728
  • Country: ca
Re: 6GSps FPGA to Memory Handling...
« Reply #28 on: April 23, 2018, 03:12:03 pm »
Why not? Acquiring 4 ADC channels to DDR3 memory doesn't require a lot of logic. The limiting factor is the number of pins. What Artix have you used on your last board? What was resource utilization for your MIG?
Actually it does. I don't have exact numbers right now, but I remember it took over 10k LUTs and 10k+ FFs just for x16 controller. Another thing is since you can only run the memory at 400 MHz (speed grade -3 devices are hard to find in stock, so I tend to avoid them), you will need to have a wide memory bus to get the kind of BW you need (x16 only gives 1600 MB/s), but for that you need to use at least 484 package - and that package you will have to use logic level translators to get SPI flash to work for booting up FPGA. Unfortunately due to the way IO banks are bonded out on Atrix the only way to make x64 DDR3 using pins on a right column is to use A200 device in 676 package, but there are some gotchas with using banks 16 and 35 if you want to use MGTs at full 6G speed as well.

Of course, if there's some sort of data processing besides acquisition, this should be accounted for.
That's another Terra incognita that tends to bite you in the behind if you're not careful. It's better to have oversized device during development because debugging aids like ILA also consumes some FPGA resources.

Offline asmi

  • Super Contributor
  • ***
  • Posts: 2728
  • Country: ca
Re: 6GSps FPGA to Memory Handling...
« Reply #29 on: April 23, 2018, 03:16:56 pm »
today, what is unachieveable is the sw license cost :palm: people just know how to get money. its understandable for FPGA development tools because they have to cover R&D and production factory, but look at things like EM Solver, the cost is just simply nuts... offtopic.
Xilinx free license gives you access to a lot of FPGAs - entire Spartan-7 and Artix-7 lines, some Kintex (K70 and K160), some Zynqs(030 and below) and also some UltraScale/UltraScale+ devices (including about half of Zynq US+ MPSoCs lineup). On top of that there are about 300 IPs available with that free license.
So you can do quite a bit with all that. This is why I've chosen to use Xilinx devices - they are much more friendly to hobbyists.
« Last Edit: April 23, 2018, 03:18:37 pm by asmi »
 
The following users thanked this post: Mechatrommer

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14297
  • Country: fr
Re: 6GSps FPGA to Memory Handling...
« Reply #30 on: April 23, 2018, 03:18:10 pm »
We already had digital scopes with 8 or 16 gsa/s almost 20 years ago (see the DDA series of Lecroy scopes for instance) and this is how that was done.
exactly except they are proprietary, and the components cost back then i believe is unachieveably expensive by hobbiests during the time...

Well, actually, back then, even developing on microcontrollers was expensive. This is how companies like Microchip changed it all for hobbyists.

Old Lecroy scopes (80's-90's) have 68k CPUs. Very slow by today's standards, but at the time, a 68030 dev board was very expensive, so kinda outside the league of most hobbyists. I'm not even talking about fast ADCs.

today, what is unachieveable is the sw license cost :palm: people just know how to get money. its understandable for FPGA development tools because they have to cover R&D and production factory, but look at things like EM Solver, the cost is just simply nuts... offtopic.

Yes. Costs have shifted towards software, but a lot of recent digital scopes (except the low-end ones) still use some proprietary ASICs.

You want crazy costs? Look at the costs for microelectronics development (Cadence, Synopsys, Mentor...) Often over $100 000 with additional recurrent maintenance costs to top it off.

 

Offline NorthGuy

  • Super Contributor
  • ***
  • Posts: 3137
  • Country: ca
Re: 6GSps FPGA to Memory Handling...
« Reply #31 on: April 23, 2018, 03:28:59 pm »
for now, the paid $4000 quartus SW is not justifiable for a one-off that probably never see the light. so using an FPGA that doesnt require software license is much preferable. earlier i said the FPGA probably will perform FFT as well, but if this means requiring larger FPGA with multi thousand dollar license, then i think it should be beneficial in term of cost to off load the FFT to MCU processing, Arm V7 1GHz quad core or the latest 64bits Arm A53 can be hard at cheap, or maybe even adding 2nd FPGA specialized in FFT alone, this... mmmv (my milage may vary). now i'm not sure on licensing scheme of other brand like the mentioned Artix KIntex etc, but if the cost of the license is near 4 digits cost then i think they have to be discarded from the list, for now while the money is still peanut small...

Xilinx's Vivado is free for all Artixes and for smaller (70 and 160) Kintexes. The free package is called WebPack. You do need to register and apply for the free license online. Also, the free version collect usage data (called WebTack) which you cannot turn off.

Depending on the size and speed of your FFT it may take more resources or less resources. You can download Vivado from Xilinx's Web site, install it, create a project for a big FPGA (I usually use A200T frown upon here), create your design, simulate, look at the resource utilization. This will give you the basis to decide how big a chip you need.

Using external MCU will require passing data to MCU, which may be difficult. Xilinx has combined chips - ARM + FPGA. They're called Zynq. They facilitate communications between ARM and FPGA. Another possibility is to build soft CPU (Xilinx's has a core called MicroBlaze which is decent).

However, FFT is better done in FPGA fabric - they have DSP blocks which can do MADD at roughly 500 MHz (if properly pipelined). Even the smallest Artix has 40 DSP blocks. So, imagine all of them working in parallel at 500 MHz. This easily beats any MCU.

 
The following users thanked this post: Mechatrommer

Online MechatrommerTopic starter

  • Super Contributor
  • ***
  • Posts: 11535
  • Country: my
  • reassessing directives...
Re: 6GSps FPGA to Memory Handling...
« Reply #32 on: April 23, 2018, 04:59:35 pm »
Xilinx's Vivado is free for all Artixes and for smaller (70 and 160) Kintexes...
i just registered in Xilinx website. when i tried to download Vivado, Xilinx required me to fill my details (Malaysia) due to "U.S. Government Export Approval" and i got this... "We cannot fulfill your request as your account has failed export compliance verification." great!! we are a big threat to the USA :palm:
Nature: Evolution and the Illusion of Randomness (Stephen L. Talbott): Its now indisputable that... organisms “expertise” contextualizes its genome, and its nonsense to say that these powers are under the control of the genome being contextualized - Barbara McClintock
 

Offline NorthGuy

  • Super Contributor
  • ***
  • Posts: 3137
  • Country: ca
Re: 6GSps FPGA to Memory Handling...
« Reply #33 on: April 23, 2018, 05:34:09 pm »
Xilinx's Vivado is free for all Artixes and for smaller (70 and 160) Kintexes...
i just registered in Xilinx website. when i tried to download Vivado, Xilinx required me to fill my details (Malaysia) due to "U.S. Government Export Approval" and i got this... "We cannot fulfill your request as your account has failed export compliance verification." great!! we are a big threat to the USA :palm:

I certainly don't know US export laws, but this might have been a bug or some sort of spelling problem as well.

See https://www.xilinx.com/support/answers/44043.html

 

Online Berni

  • Super Contributor
  • ***
  • Posts: 4922
  • Country: si
Re: 6GSps FPGA to Memory Handling...
« Reply #34 on: April 23, 2018, 06:42:12 pm »
As far as i see this project is not going to be cheap in any way you look at it.

The way to go certainly is parallel. You can only increase the clock speed up to a point before the chips become rather exotic and ridiculously expensive. DDR3 memory certainly is very quick when accessed in large chunks. I would think it makes sense to split up the load between multiple FPGAs so that you can make use of cheap small versions of them and since you are starting off with multiple interleaved ADCs its easy to split them up between multiple FPGAs to take the data.

Wide memory buses are the answer. Modern PCs use this a lot as dual channel memory is pretty much standard to get a pair of 64bit DIMMs to be 128bit wide. Some big CPUs even have quad channel memory to get you a 256bit bus to RAM. Once you get to graphics cards that tend to have even more hunger for bandwidth things can get to 384bit memory buses. Recently GPU manufacturers started putting RAM chips in the same package as the GPU and wire bonding the ram to it, this got them a truly ridiculous 3072 bit bus to the RAM.

You don't necessarily have to buy the dev tools off the bat tho. All the major manufacturers let you program the smaller chips with the free version. But they will offten want you to buy the software to get a lot of the nicer soft IP such as soft DDR memory controllers, FFT blocks etc. However as far as i know Altera gives you all of this for evaluation where using this IP will make your binary file time limited. You can program your FPGA with it just fine but after about 1 to 2 hours the FPGA will deliberately crash itself when the timer runs out.
 

Online MechatrommerTopic starter

  • Super Contributor
  • ***
  • Posts: 11535
  • Country: my
  • reassessing directives...
Re: 6GSps FPGA to Memory Handling...
« Reply #35 on: April 24, 2018, 08:53:55 am »
i just realized this discussion only lingers among 2 names, Altera and Xilinx. why these are the only 2 names that produce FPGA? isnt there anyone else thats probably cheaper and SW IDE can be easily installed at cheap or free? :(
Nature: Evolution and the Illusion of Randomness (Stephen L. Talbott): Its now indisputable that... organisms “expertise” contextualizes its genome, and its nonsense to say that these powers are under the control of the genome being contextualized - Barbara McClintock
 

Offline daveshah

  • Supporter
  • ****
  • Posts: 356
  • Country: at
    • Projects
Re: 6GSps FPGA to Memory Handling...
« Reply #36 on: April 24, 2018, 10:14:55 am »
Lattice ECP5 is cheaper, but not realistically going to handle 6Gsps data flow to DDR3. Also you have to pay for the software if you need one with the transceivers.
 
The following users thanked this post: Mechatrommer

Offline ali_asadzadeh

  • Super Contributor
  • ***
  • Posts: 1896
  • Country: ca
Re: 6GSps FPGA to Memory Handling...
« Reply #37 on: April 24, 2018, 12:31:07 pm »
God has invented torrents ages ago! ;) Just saying

If this is a hobby project and a one off, maybe that would be ok to use torrents, but if it's a commercial project then the license cost would be nothing :)
ASiDesigner, Stands for Application specific intelligent devices
I'm a Digital Expert from 8-bits to 64-bits
 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14297
  • Country: fr
Re: 6GSps FPGA to Memory Handling...
« Reply #38 on: April 24, 2018, 01:02:05 pm »
Lattice ECP5 is cheaper, but not realistically going to handle 6Gsps data flow to DDR3.

Probably not. The  ECP5-5G has 5 Gbps SERDES, which is not bad.
There is a dev kit (ECP5-5G Versa), with a free 1-year licence. The board has DDR3 RAM, and apparently the max freq you can get out of it is 800 MHz, which gets you 1.6 GBytes/s since  its data bus is 16-bit.
I'm not sure what max data bus width their DDR3 controller IP supports.

Also you have to pay for the software if you need one with the transceivers.

Yes, but last I checked, the licence cost was $99/yr. It's definitely affordable.
 

Online MechatrommerTopic starter

  • Super Contributor
  • ***
  • Posts: 11535
  • Country: my
  • reassessing directives...
Re: 6GSps FPGA to Memory Handling...
« Reply #39 on: April 24, 2018, 03:00:19 pm »
God has invented torrents ages ago! ;) Just saying
If this is a hobby project and a one off, maybe that would be ok to use torrents, but if it's a commercial project then the license cost would be nothing :)
torrent is invented by man, based on Laws created by God. anyway, torrent is of no good without a license provided.. ;) i tried to download from another site https://www.cmc.ca same thing happened and then i emailed them to beg, the reply is, "its only for canadian academy" gosh thats racist.

Lattice ECP5 is cheaper, but not realistically going to handle 6Gsps data flow to DDR3.
Probably not. The  ECP5-5G has 5 Gbps SERDES, which is not bad.
i'm having trouble searching for their SW name from the website. what is it again? please.
Nature: Evolution and the Illusion of Randomness (Stephen L. Talbott): Its now indisputable that... organisms “expertise” contextualizes its genome, and its nonsense to say that these powers are under the control of the genome being contextualized - Barbara McClintock
 

Offline daveshah

  • Supporter
  • ****
  • Posts: 356
  • Country: at
    • Projects
Re: 6GSps FPGA to Memory Handling...
« Reply #40 on: April 24, 2018, 03:52:53 pm »
The software is Lattice Diamond.

MicroSemi PolarFire might also be worth a look.
 
The following users thanked this post: Mechatrommer

Offline daveshah

  • Supporter
  • ****
  • Posts: 356
  • Country: at
    • Projects
Re: 6GSps FPGA to Memory Handling...
« Reply #41 on: April 24, 2018, 03:55:28 pm »
It looks like the ECP5 DDR3 controller can do up to a 64-bus at 800Mbps. Which is 6.4Byte/s.
 

Online Berni

  • Super Contributor
  • ***
  • Posts: 4922
  • Country: si
Re: 6GSps FPGA to Memory Handling...
« Reply #42 on: April 24, 2018, 04:12:03 pm »
The FPGA market is in the hands of only a few vendors and they all seam to have the same sort of business model. They all offer a large range of chips from tiny cheap ones to ridiculously expensive but also incredibly powerful chips. They all give you the software for free with the cheap chips but will attempt to sell it to you for >1000$ otherwise. They all offer you some expensive but very useful IP blocks (A lot of them you get for free if you buy that expensive software)

Altera and Xilinx are simply the two biggest players in the market. They make the highest performance chips and they have the best software tools (Tho Altera IDE does some things better while Xilinx does other things better, overall they are both good, but not perfect)

Then people like Lattice or Microsemi are sort of the cheap brand ones. The chips cost less but the software is not nearly as good (Tho the software can still be just as expensive as Altera or Xilinx).

In general its a good idea to stick to LVDS output ADCs since you can run receive 800Mbit/s per lane on 20 or more lanes on most cheap FPGAs out there. The chips with transceivers get expensive really quick. You can avoid paying for the DDR3 controller license if you get a chip with a hardIP DDR3 controller(At least thats how it works at Altera for Cyclone V), but as said before you might need 2 to 4 FPGAs to get enough memory bandwidth.

If this is a one off project tho it might make sense to look into any existing hardware for this such a PXI mainframe ADC digitizer cards that can do such speeds.

 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14297
  • Country: fr
Re: 6GSps FPGA to Memory Handling...
« Reply #43 on: April 24, 2018, 04:18:42 pm »
 
The following users thanked this post: Mechatrommer

Offline NorthGuy

  • Super Contributor
  • ***
  • Posts: 3137
  • Country: ca
Re: 6GSps FPGA to Memory Handling...
« Reply #44 on: April 24, 2018, 05:11:20 pm »
... but for that you need to use at least 484 package - and that package you will have to use logic level translators to get SPI flash to work for booting up FPGA.

Silly, isn't it? It's even funnier if you want to configure the FPGA from MCU as SPI slave. This uses one pin (one pin!) in bank 14, and it mandates that all the configuration and system pins must be at 1.5V. What could they possibly think when they designed it this way? I use level shifters - $5 is nothing compared to upgrading FPGA.
 

Offline asmi

  • Super Contributor
  • ***
  • Posts: 2728
  • Country: ca
Re: 6GSps FPGA to Memory Handling...
« Reply #45 on: April 24, 2018, 11:48:55 pm »
Silly, isn't it? It's even funnier if you want to configure the FPGA from MCU as SPI slave. This uses one pin (one pin!) in bank 14, and it mandates that all the configuration and system pins must be at 1.5V. What could they possibly think when they designed it this way? I use level shifters - $5 is nothing compared to upgrading FPGA.
For QSPI there are 4 data pins (D00-D03) and chip select. Using single-pin SPI is incredibly slow, boot time is going to be seconds even for small devices.
The pinout design implies that banks 14 and 15 are used for configuration, and in case of parallel flash you will need a lot of these pins. Remember it was designed before QSPI flash has become a commodity, if you take a look at configuration-related pins in newer UltraScale/UltraScale+ devices, you will see that all QSPI-related pins are a part of config bank 0, so it doesn't force any particular voltage levels for any of user I/O banks (see here, page 27). So it's hard to blame them for failing to predict where market will go almost a decade later.
« Last Edit: April 24, 2018, 11:51:49 pm by asmi »
 

Offline NorthGuy

  • Super Contributor
  • ***
  • Posts: 3137
  • Country: ca
Re: 6GSps FPGA to Memory Handling...
« Reply #46 on: April 25, 2018, 12:17:19 am »
For QSPI there are 4 data pins (D00-D03) and chip select. Using single-pin SPI is incredibly slow, boot time is going to be seconds even for small devices.

SPI flash can typically run at 50 MHz which is 0.3 sec for full bitstream for A50T. Good enough for me. Using SPI from MCU at 25 MHz is still less than a second - faster than JTAG. Even at 10 MHz it wouldn't be that bad.

We're hijacking a thread again ... :)
 

Offline asmi

  • Super Contributor
  • ***
  • Posts: 2728
  • Country: ca
Re: 6GSps FPGA to Memory Handling...
« Reply #47 on: April 25, 2018, 01:44:54 am »
SPI flash can typically run at 50 MHz which is 0.3 sec for full bitstream for A50T. Good enough for me. Using SPI from MCU at 25 MHz is still less than a second - faster than JTAG. Even at 10 MHz it wouldn't be that bad.
There are scenarios where this is unacceptable. For example PCI Express mandates 100 ms power-good-to-ready time limit. This is not an easy requirement to fulfill - especially as you will need to use larger device for PCI-E endpoint because it's quite logic-heavy ("AXI Memory Mapped to PCI Express" IP on one of my test designs is ~15k LUTs, 12k FF  and 19 BRAMs).

We're hijacking a thread again ... :)
Disagree. Since the project at hand practically mandates x64 memory bus, figuring out if 484 package can be made to work is important, especially since pinout layout of smaller 676 devices (A75 and A100) still force using configuration banks as only two banks are bonded out on a left column. So this is important consideration.
« Last Edit: April 25, 2018, 01:48:51 am by asmi »
 

Offline NiHaoMike

  • Super Contributor
  • ***
  • Posts: 8973
  • Country: us
  • "Don't turn it on - Take it apart!"
    • Facebook Page
Re: 6GSps FPGA to Memory Handling...
« Reply #48 on: April 25, 2018, 05:29:25 am »
SPI flash can typically run at 50 MHz which is 0.3 sec for full bitstream for A50T. Good enough for me. Using SPI from MCU at 25 MHz is still less than a second - faster than JTAG. Even at 10 MHz it wouldn't be that bad.
There are scenarios where this is unacceptable. For example PCI Express mandates 100 ms power-good-to-ready time limit. This is not an easy requirement to fulfill - especially as you will need to use larger device for PCI-E endpoint because it's quite logic-heavy ("AXI Memory Mapped to PCI Express" IP on one of my test designs is ~15k LUTs, 12k FF  and 19 BRAMs).
Many years ago, I dealt with a FPGA PCIe card at work where the instructions included holding down the reset button for a second or so when powering up the PC to allow the FPGA to boot. I suggested adding a capacitor across the reset button to provide the needed delay.
Cryptocurrency has taught me to love math and at the same time be baffled by it.

Cryptocurrency lesson 0: Altcoins and Bitcoin are not the same thing.
 

Online MechatrommerTopic starter

  • Super Contributor
  • ***
  • Posts: 11535
  • Country: my
  • reassessing directives...
Re: 6GSps FPGA to Memory Handling...
« Reply #49 on: April 25, 2018, 05:48:34 am »
We're hijacking a thread again ... :)
hijack granted. you may proceed as you like ;D

We're hijacking a thread again ... :)
Disagree. Since the project at hand practically mandates x64 memory bus, figuring out if 484 package can be made to work is important, especially since pinout layout of smaller 676 devices (A75 and A100) still force using configuration banks as only two banks are bonded out on a left column. So this is important consideration.
agree, 32bits @ 1.6GHz, 64bits @ 800MHz and so on. both number of pins and clock speed need to be considered from every aspects. i may not fully understand what you people discussing here, but who knows this may be valuable reference later. the golden advice so far thats applicable to me is, download the IDE and simulate the program for the FPGA.
Nature: Evolution and the Illusion of Randomness (Stephen L. Talbott): Its now indisputable that... organisms “expertise” contextualizes its genome, and its nonsense to say that these powers are under the control of the genome being contextualized - Barbara McClintock
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf