Author Topic: Streaming data FPGA interface -> off the shelf?  (Read 3555 times)

0 Members and 1 Guest are viewing this topic.

Offline drwho9437

  • Regular Contributor
  • *
  • Posts: 52
Streaming data FPGA interface -> off the shelf?
« on: January 08, 2022, 09:40:05 pm »
I am looking for good solutions to provide steaming data back to a computer from a digital system.

I am aware of Opal Kelly's offerings, but feel the value proposition for me isn't that great. I don't need a big FPGA or lots of memory really and of course they realize the value of their software and charge a reasonable industrial rate for their stuff. At work that is fine but not for home stuff.

I have thought about using the FX-3 USB 3 chip myself and making my own but of course the development time would be high. It is also only BGA. FTDI has some USB 3 FIFO chips that look possible too but again lots of driver work and more to get it going.

So bottom line: what are the options out there? Seems like this should be a solved problem with a low margin off the shelf option -> a fat low latency pipe from a computer to an FPGA.

What do you use?
 

Offline Foxxz

  • Regular Contributor
  • *
  • Posts: 83
  • Country: us
Re: Streaming data FPGA interface -> off the shelf?
« Reply #1 on: January 08, 2022, 10:16:28 pm »
I was going to suggest the FX3 until you ruled it out. Not sure how much driver work needs to be done as I've seen these paired with plenty of FPGAs.

I might suggest an FTDI FT2232H which is USB2 but the driver work would be little to nothing.

There are FPGA PCIE bridges out there for which the driver is free but you license the IP to use in your design. If you need it to be detachable I have gotten it to work over lightning ports.
 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 9545
  • Country: fr
Re: Streaming data FPGA interface -> off the shelf?
« Reply #2 on: January 08, 2022, 10:19:01 pm »
I might suggest an FTDI FT2232H which is USB2 but the driver work would be little to nothing.

Yes, drivers are available for all major platforms, plus a decent API.
They absolutely have USB 3 SS chips these days. FT6xx series. Not only are they no work for the drivers, but they are also significantly cheaper than the FX3.

 

Offline drwho9437

  • Regular Contributor
  • *
  • Posts: 52
Re: Streaming data FPGA interface -> off the shelf?
« Reply #3 on: January 08, 2022, 10:27:56 pm »
I was going to suggest the FX3 until you ruled it out. Not sure how much driver work needs to be done as I've seen these paired with plenty of FPGAs.

I might suggest an FTDI FT2232H which is USB2 but the driver work would be little to nothing.

There are FPGA PCIE bridges out there for which the driver is free but you license the IP to use in your design. If you need it to be detachable I have gotten it to work over lightning ports.

I'm not ruling out the FX3. It seems like a lot of people have selected it so it must have something going for it but I'm not that strong on USB driver classes. FTDI has a USB 3 chip that is a hard wired FIFO of some kind in a somewhat easier to solder QFN than the FX3 BGA. Not sure if there is a serious drawback to the FTDI solution.
 

Offline drwho9437

  • Regular Contributor
  • *
  • Posts: 52
Re: Streaming data FPGA interface -> off the shelf?
« Reply #4 on: January 08, 2022, 10:30:25 pm »
I might suggest an FTDI FT2232H which is USB2 but the driver work would be little to nothing.

Yes, drivers are available for all major platforms, plus a decent API.
They absolutely have USB 3 SS chips these days. FT6xx series. Not only are they no work for the drivers, but they are also significantly cheaper than the FX3.

Have you use the FT6xx series? I looked at the docs but it wasn't clear exactly what the PC interface would look like via the standard driver. Development board is about 100 dollars and I considered getting it and the cyclone dev board it mates to just to figure out how to use everything that way. Probably sell the cyclone board after I understand everything was my first line of thinking.

Wonder if there are other options with higher levels of integration though like the Opel Kelly stuff but less expensive.
 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 9545
  • Country: fr
Re: Streaming data FPGA interface -> off the shelf?
« Reply #5 on: January 08, 2022, 10:35:42 pm »
I might suggest an FTDI FT2232H which is USB2 but the driver work would be little to nothing.

Yes, drivers are available for all major platforms, plus a decent API.
They absolutely have USB 3 SS chips these days. FT6xx series. Not only are they no work for the drivers, but they are also significantly cheaper than the FX3.

Have you use the FT6xx series? I looked at the docs but it wasn't clear exactly what the PC interface would look like via the standard driver.

Not yet, but you can read the doc here: https://ftdichip.com/wp-content/uploads/2020/08/AN_379-D3xx-Programmers-Guide-1.pdf

I have used the USB 2 HS parts a lot though, with their D2XX drivers, and the API looks absolutely similar.
Pretty easy to use.
 

Offline evb149

  • Super Contributor
  • ***
  • Posts: 2370
  • Country: us
Re: Streaming data FPGA interface -> off the shelf?
« Reply #6 on: January 08, 2022, 10:59:49 pm »
You mentioned "home stuff" prices.

So if you're not looking for something industrial and embeddable in production why not consider the aforementioned (Cypress, FTDI) USB3 bridges *developer kits*?  They're "ready to do" except for wiring interface to your target system (which itself is non-trivial for high speed signal integrity if you have NN high speed digital signals to attach, so making an interface PCB + cable assembly is a good idea).

Also why not just any common FPGA board which includes a gigabit ethernet interface?  There are several out there for moderate prices and then you have almost the simplest possible streaming connection to a PC if you send UDP or such like packets.  And there are free IP cores / codes etc. to do it.  If you spend more you can get ones with multiple 1Gb Ethernet interfaces or something like 10Gb/s ethernet interfaces copper or with a SFP cage for your transceiver of choice.

If you look at sigrok.org and "supported hardware" they have a list of "logic analyzer" type digital signal acquisition boards that are supported by the sigrok open source software for digital state capture at the specified rates and port counts.  Among those are the typical sort of inexpensive "hobbyist" boards as well as some mid-range and more commercial / industrial oriented stuff.

The nice thing about ethernet though is that you can have a choice of either using several gigabit ethernet links coming from one board (if needed) or using several different interface boards each with one 1Gb/s interface and then a choice of the switch / NIC on the PC side that can be 10Gb, 5Gb, 2.5Gb, 1Gb per port with N ports so the scalability of the solution in terms of multiplicity of targets as well as multiplicity of bandwidth is good.  And you can get cheap PC hardware and switches and even cheap-ish devkits with ethernet & FPGAs.

Then there's PCI where there are again some relatively low cost available PCI/PCIE devkits which can be used and there are IP cores / sample designs out there that make it easier to use.

 

Offline asmi

  • Super Contributor
  • ***
  • Posts: 2095
  • Country: ca
Re: Streaming data FPGA interface -> off the shelf?
« Reply #7 on: January 08, 2022, 11:14:33 pm »
Have you use the FT6xx series? I looked at the docs but it wasn't clear exactly what the PC interface would look like via the standard driver. Development board is about 100 dollars and I considered getting it and the cyclone dev board it mates to just to figure out how to use everything that way. Probably sell the cyclone board after I understand everything was my first line of thinking.
I have used FT601 with Artix-7 FPGA, and was very happy with the results. They provide a library for PC side (along with drivers of course) which you can use in C/C++ or .NET, I used the latter and was able to reach about 325 MBytes/s of sustained data bandwidth (that's the real speed your application is receiving data at after all overheards are accounted for). On FPGA side it's just a simple FIFO-style interface. I used it on my own custom board, so not sure how it works on their devboards. The only real challenge is figuring out a way to route so many parallel traces to FPGA without getting crosstalk issues. If you don't need that much of a bandwidth, you might want to use FT600 instead as it uses only 16bit wide data bus (as opposed to 32bit for FT601).

This is as "off the shelf" as it goes in this business, FPGA side interface is not much more complex than interfacing with internal FIFO, you can choose the frequency it runs at between 66 and 100 MHz (I used the latter, the former will reduce available bandwidth), and it's just a few lines of code on the PC side (at least using their .NET library).

Ethernet is much more complicated solution than FT60x, not to mention if you do need something close to 2.5 Gbps which they provide. PCI Express is even worse as it requires using dedicated multi-gigabit transceivers, as well as writing a driver on the PC side. Xilinx provides a free IP and a driver, but it's only for Linux. other vendors charge big $$$ even for IP.
« Last Edit: January 09, 2022, 12:04:13 am by asmi »
 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 9545
  • Country: fr
Re: Streaming data FPGA interface -> off the shelf?
« Reply #8 on: January 08, 2022, 11:30:03 pm »
For the FT232H/2232H, the synchronous FIFO mode is a "master" FIFO - the FT part provides the clock. So either you can use the same clock throughout your design (but beware this clock will stop when the communication is closed), or you need to synchronize between clock domains. Not a big deal, just something to keep in mind.

I'm not sure about the FT6xx - I think they act as FIFO slave, thus you provide the clock, which makes things a little easier - but asmi can confirm.
 

Offline langwadt

  • Super Contributor
  • ***
  • Posts: 3089
  • Country: dk
Re: Streaming data FPGA interface -> off the shelf?
« Reply #9 on: January 08, 2022, 11:34:52 pm »
I might suggest an FTDI FT2232H which is USB2 but the driver work would be little to nothing.

Yes, drivers are available for all major platforms, plus a decent API.

yes, good for about 40MB/s in fifo mode and you can use the second interface to configure the fpga
 

Offline asmi

  • Super Contributor
  • ***
  • Posts: 2095
  • Country: ca
Re: Streaming data FPGA interface -> off the shelf?
« Reply #10 on: January 09, 2022, 12:02:59 am »
I'm not sure about the FT6xx - I think they act as FIFO slave, thus you provide the clock, which makes things a little easier - but asmi can confirm.
No, it's a master too (meaning the chip provides the clock), but you are likely want to have FIFOs on FPGA side as well to cross into whatever clock domain your design is running at, so it's not a big deal.

Offline voltsandjolts

  • Supporter
  • ****
  • Posts: 1687
  • Country: gb
Re: Streaming data FPGA interface -> off the shelf?
« Reply #11 on: January 09, 2022, 03:14:09 pm »
cross-ref to another thread and an interesting post about fx2pipe for cypress fx2 (never tried it myself).
 

Offline NorthGuy

  • Super Contributor
  • ***
  • Posts: 2786
  • Country: ca
Re: Streaming data FPGA interface -> off the shelf?
« Reply #12 on: January 09, 2022, 06:53:36 pm »
It is huge difference in speed between USB 2 and USB 3. The USB 2 solutions are abundant. USB 3 options are very scarce. All I could find is Cypress FX3 and FT601. FX3 is expensive, complex, and poorly documented. So, I made a prototype board with FT601. It worked well. It took me less than a day to begin communicating with FPGA through their library.

The FT601 FPGA FIFO interface is rather straightforward. The timing requirements in their documentation are very strange. They require huge hold time (4.8 ns) which is absolutely impossible to meet. They have an XDL example where they "forgot" to put minus - they have "set_output_delay -min 4.8" instead of "set_output_delay -min -4.8". This certainly passes timing analysis every time. But if you put the "minus" in, it's utterly impossible. Even though I had some help from the trace delay (400 ps), I could only meet 2.3 ns hold. This worked well at 100 MHz, buf failed at 66 MHz. From which I concluded that 4.8 ns is either bogus, or reflects only the worst case (66 MHz at 1.8V).

The DLL works, but is cumbersome, loads slow (300 ms), and is somewhat buggy. FT601 doesn't really do FIFO, rather block transfers with FIFO interface to FPGA. If you do simple block transfers with blocks of the fixed size everything works. With variable block size it becomes less reliable. Sometimes the read function returns with data size less than requested (contrary to the docs). If application quits without reading all the requested data, next time the DLL loads, it can read leftover data from the previous start. Sometimes it simply hangs and starts returning errors on all reads.

I couldn't get full speed from the DLL (381 MB/s), even when reading one long block. No matter what you do there are delays between blocks, so you realistically get 320-340 MB/s. While analyzing delays on the FPGA side I noticed that gaps between blocks are typically small, but sometimes there are longer delays as well. The nature of these delays is unknown. May be FT601 limitations. May be DLL problems.

It should be possible to figure out the USB protocol directly and get rid of the DLL, fixing the bugs, and possibly improving performance. I was going to explore this opportunity, but by that time the cheap Xilinx FPGAs turned into unobtainium and the project was scrapped.
 

Offline drwho9437

  • Regular Contributor
  • *
  • Posts: 52
Re: Streaming data FPGA interface -> off the shelf?
« Reply #13 on: January 09, 2022, 11:07:31 pm »
You mentioned "home stuff" prices.

So if you're not looking for something industrial and embeddable in production why not consider the aforementioned (Cypress, FTDI) USB3 bridges *developer kits*?  They're "ready to do" except for wiring interface to your target system (which itself is non-trivial for high speed signal integrity if you have NN high speed digital signals to attach, so making an interface PCB + cable assembly is a good idea).

Also why not just any common FPGA board which includes a gigabit ethernet interface?  There are several out there for moderate prices and then you have almost the simplest possible streaming connection to a PC if you send UDP or such like packets.  And there are free IP cores / codes etc. to do it.  If you spend more you can get ones with multiple 1Gb Ethernet interfaces or something like 10Gb/s ethernet interfaces copper or with a SFP cage for your transceiver of choice.

If you look at sigrok.org and "supported hardware" they have a list of "logic analyzer" type digital signal acquisition boards that are supported by the sigrok open source software for digital state capture at the specified rates and port counts.  Among those are the typical sort of inexpensive "hobbyist" boards as well as some mid-range and more commercial / industrial oriented stuff.

The nice thing about ethernet though is that you can have a choice of either using several gigabit ethernet links coming from one board (if needed) or using several different interface boards each with one 1Gb/s interface and then a choice of the switch / NIC on the PC side that can be 10Gb, 5Gb, 2.5Gb, 1Gb per port with N ports so the scalability of the solution in terms of multiplicity of targets as well as multiplicity of bandwidth is good.  And you can get cheap PC hardware and switches and even cheap-ish devkits with ethernet & FPGAs.

Then there's PCI where there are again some relatively low cost available PCI/PCIE devkits which can be used and there are IP cores / sample designs out there that make it easier to use.

I have certainly thought about going the Ethernet UDP route for a long time but I have dithered on it. PCIE is a no, I want the box on a cable. I don't think the dev kits are a problem I just want to know the market before I invest the time. Still not sure if there are any higher level of integration other than Opal Kelly. I agree though GigE would be lower cost and in some ways simpler.
 

Offline drwho9437

  • Regular Contributor
  • *
  • Posts: 52
Re: Streaming data FPGA interface -> off the shelf?
« Reply #14 on: January 09, 2022, 11:10:15 pm »
Have you use the FT6xx series? I looked at the docs but it wasn't clear exactly what the PC interface would look like via the standard driver. Development board is about 100 dollars and I considered getting it and the cyclone dev board it mates to just to figure out how to use everything that way. Probably sell the cyclone board after I understand everything was my first line of thinking.
I have used FT601 with Artix-7 FPGA, and was very happy with the results. They provide a library for PC side (along with drivers of course) which you can use in C/C++ or .NET, I used the latter and was able to reach about 325 MBytes/s of sustained data bandwidth (that's the real speed your application is receiving data at after all overheards are accounted for). On FPGA side it's just a simple FIFO-style interface. I used it on my own custom board, so not sure how it works on their devboards. The only real challenge is figuring out a way to route so many parallel traces to FPGA without getting crosstalk issues. If you don't need that much of a bandwidth, you might want to use FT600 instead as it uses only 16bit wide data bus (as opposed to 32bit for FT601).

This is as "off the shelf" as it goes in this business, FPGA side interface is not much more complex than interfacing with internal FIFO, you can choose the frequency it runs at between 66 and 100 MHz (I used the latter, the former will reduce available bandwidth), and it's just a few lines of code on the PC side (at least using their .NET library).

Ethernet is much more complicated solution than FT60x, not to mention if you do need something close to 2.5 Gbps which they provide. PCI Express is even worse as it requires using dedicated multi-gigabit transceivers, as well as writing a driver on the PC side. Xilinx provides a free IP and a driver, but it's only for Linux. other vendors charge big $$$ even for IP.

Very helpful advice. I might try to build a custom Max10 + FT600 board and see how far I get.
 

Offline drwho9437

  • Regular Contributor
  • *
  • Posts: 52
Re: Streaming data FPGA interface -> off the shelf?
« Reply #15 on: January 09, 2022, 11:16:53 pm »
It is huge difference in speed between USB 2 and USB 3. The USB 2 solutions are abundant. USB 3 options are very scarce. All I could find is Cypress FX3 and FT601. FX3 is expensive, complex, and poorly documented. So, I made a prototype board with FT601. It worked well. It took me less than a day to begin communicating with FPGA through their library.

The FT601 FPGA FIFO interface is rather straightforward. The timing requirements in their documentation are very strange. They require huge hold time (4.8 ns) which is absolutely impossible to meet. They have an XDL example where they "forgot" to put minus - they have "set_output_delay -min 4.8" instead of "set_output_delay -min -4.8". This certainly passes timing analysis every time. But if you put the "minus" in, it's utterly impossible. Even though I had some help from the trace delay (400 ps), I could only meet 2.3 ns hold. This worked well at 100 MHz, buf failed at 66 MHz. From which I concluded that 4.8 ns is either bogus, or reflects only the worst case (66 MHz at 1.8V).

The DLL works, but is cumbersome, loads slow (300 ms), and is somewhat buggy. FT601 doesn't really do FIFO, rather block transfers with FIFO interface to FPGA. If you do simple block transfers with blocks of the fixed size everything works. With variable block size it becomes less reliable. Sometimes the read function returns with data size less than requested (contrary to the docs). If application quits without reading all the requested data, next time the DLL loads, it can read leftover data from the previous start. Sometimes it simply hangs and starts returning errors on all reads.

I couldn't get full speed from the DLL (381 MB/s), even when reading one long block. No matter what you do there are delays between blocks, so you realistically get 320-340 MB/s. While analyzing delays on the FPGA side I noticed that gaps between blocks are typically small, but sometimes there are longer delays as well. The nature of these delays is unknown. May be FT601 limitations. May be DLL problems.

It should be possible to figure out the USB protocol directly and get rid of the DLL, fixing the bugs, and possibly improving performance. I was going to explore this opportunity, but by that time the cheap Xilinx FPGAs turned into unobtainium and the project was scrapped.

Very interesting hopefully you will unscrap it if the chip shortage eases. I think a well documented reference big pipe OSH design could benefit people a lot. In my mind something like a Max10 base able to pull some LVDS lines to some DAC/ADC would be a thing that would be pretty enabling.

I didn't completely follow your timing issue but I guess I would figure it out when I get there. I haven't made a comparison between the FT6xx and the older FT FIFO stuff. Other than the PHY is there much difference in using them? I'd rather standardize my design on something future looking even if the older USB 2 FIFO is a tiny bit easier. If it is a lot easier perhaps I should rethink.
 

Offline NorthGuy

  • Super Contributor
  • ***
  • Posts: 2786
  • Country: ca
Re: Streaming data FPGA interface -> off the shelf?
« Reply #16 on: January 11, 2022, 03:12:08 am »
I didn't completely follow your timing issue but I guess I would figure it out when I get there. I haven't made a comparison between the FT6xx and the older FT FIFO stuff. Other than the PHY is there much difference in using them? I'd rather standardize my design on something future looking even if the older USB 2 FIFO is a tiny bit easier. If it is a lot easier perhaps I should rethink.

USB 2.0 full speed uses 480 MHz signaling. This will give you 50 MB/s at the very best. If this is sufficient for you, I would definitely go with USB 2. I don't know what you mean by "easier", but it's certainly much cheaper, uses slower signals, saves space on PCB, and gives you more options. If you need 50 MB/s or more, then USB 2 is not fast enough for you.
 

Online Someone

  • Super Contributor
  • ***
  • Posts: 3364
  • Country: au
    • send complaints here
Re: Streaming data FPGA interface -> off the shelf?
« Reply #17 on: January 11, 2022, 03:51:01 am »
I think a well documented reference big pipe OSH design could benefit people a lot.
Pick your favorite open source SDR.

spoiler... they mostly use FX2/FX3 chips!
 

Offline laugensalm

  • Regular Contributor
  • *
  • Posts: 86
  • Country: ch
Re: Streaming data FPGA interface -> off the shelf?
« Reply #18 on: January 11, 2022, 12:34:47 pm »
One more thumb up for all the GNU radio derivatives, they offer the best FX2 code I've seen so far. Similar usable reference designs are found for FX3 *without* the flaky ThreadX framework (flaky as in introducing spurious DMA timeouts).

However, I'd definitely go with UDP over GigE these days, the software overhead for USB host-side (drivers, VID/PID handling) plus embedded firmware is way over the little more expense for the ethernet streaming logic. Especially if out of band messages are to be transmitted, Ethernet does the better job. Depending on what you're streaming, you might also rather try compressing it (DPCM, context based prediction, etc.) instead of driving your interface near the maximum possible data rate.

 

Offline voltsandjolts

  • Supporter
  • ****
  • Posts: 1687
  • Country: gb
Re: Streaming data FPGA interface -> off the shelf?
« Reply #19 on: January 11, 2022, 01:07:19 pm »
they offer the best FX2 code I've seen so far

Can you provide a link to your recommended fx2 code please?
 

Offline laugensalm

  • Regular Contributor
  • *
  • Posts: 86
  • Country: ch
Re: Streaming data FPGA interface -> off the shelf?
« Reply #20 on: January 11, 2022, 02:59:43 pm »
This is the most recent I've found (my repo clones are kinda outdated).

https://github.com/EttusResearch/uhd/tree/master/firmware/fx2

Also, you might find derived code around all 'usbjtag' related source, at least for bulk transfers. All this code should compile with the license-free SDCC, I just remember some hickups with the migration to SDCC v3.
 
The following users thanked this post: voltsandjolts

Offline Morgan127

  • Contributor
  • Posts: 27
  • Country: se
Re: Streaming data FPGA interface -> off the shelf?
« Reply #21 on: January 11, 2022, 08:33:51 pm »
I can recommend you to have a look at the Ethernet cores from FPGA-Cores.com.
These are very easy to use and you get UDP / TCP and other stuff like remote programming, logic analyzer etc.
No charge for private projects. Free to download. I use them in almost all my projects.

 

Offline NorthGuy

  • Super Contributor
  • ***
  • Posts: 2786
  • Country: ca
Re: Streaming data FPGA interface -> off the shelf?
« Reply #22 on: January 11, 2022, 09:02:27 pm »
I can recommend you to have a look at the Ethernet cores from FPGA-Cores.com.
These are very easy to use and you get UDP / TCP and other stuff like remote programming, logic analyzer etc.
No charge for private projects. Free to download. I use them in almost all my projects.

Looks like these are RMII (10/100Mbit Ethernet), which is even slower than USB 2. Various methods can be ranked by their speed as follows:

USB 3 > Gigabit Ethernet > USB 2 > 100Mbit Ethernet
« Last Edit: January 11, 2022, 09:06:38 pm by NorthGuy »
 

Offline Morgan127

  • Contributor
  • Posts: 27
  • Country: se
Re: Streaming data FPGA interface -> off the shelf?
« Reply #23 on: January 12, 2022, 05:50:23 am »
I can recommend you to have a look at the Ethernet cores from FPGA-Cores.com.
These are very easy to use and you get UDP / TCP and other stuff like remote programming, logic analyzer etc.
No charge for private projects. Free to download. I use them in almost all my projects.

Looks like these are RMII (10/100Mbit Ethernet), which is even slower than USB 2. Various methods can be ranked by their speed as follows:

USB 3 > Gigabit Ethernet > USB 2 > 100Mbit Ethernet

There are MII, RMII, GMII and RGMII so also Gigabit Ethernet.
 

Offline drwho9437

  • Regular Contributor
  • *
  • Posts: 52
Re: Streaming data FPGA interface -> off the shelf?
« Reply #24 on: January 22, 2022, 07:17:36 pm »
For industrial application GigE or other Ethernet is good but the switch is some overhead and power over Ethernet is a thing but not as simple. I am going to take a look at the FTDI 16-bit FIFO chips in a bit more detail and see how far I get I think. My preliminary pick is 10M02SCM153 + FT600 -> connector. Sometimes I don't have time to really get into this stuff so we will see how far I get. This work isn't my highest priority but its somewhere on the list.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf