Cheap solution for a fast SPI analyzer/sniffer?

[Novgorod]

I did not understand why a high sampling rate of 200 MHz is needed.

That's for a traditional logic analyzer with a free-running clock (not synchronized to the signal you measure) - the rule of thumb is that you need about 8-10x over-sampling of a digital waveform for reliable reconstruction, so using an FX3 board for 100MHz capture (or another 100MHz logic analyzer for a comparable price) just won't cut it. The 33MHz clock signal is high/low for 15ns (50% duty cycle), you can't sample it reliably with 10ns samples (huge timing jitter and the threshold can be missed).

If you want to observe the data stream and not the waveform, then 33/8=~4 MB/second.
The MCU will receive a serial signal and output it decoded as byte stream - any MC is capable of this.

It's double that, as someone mentioned. And if the MC is doing GPIO sampling, i.e. working as a logic analyzer, the above argument applies in the same way - it must be capable of very high sampling rates to reliably bit-bang the SPI protocol, and then have enough resources left to process the data (extract the 2-channel bit stream using the recorded clock channel) and re-package it into a byte stream over USB. So the MC would need the capabilities of a pretty good logic analyzer with real-time protocol decoder. I doubt any Arduino can do that.

What they can do (maybe?) is using dedicated SPI hardware for the protocol decoding (as you also suggested), but this has significant limitations when running in slave mode, i.e. locking onto an external data clock, see the link I've posted for the Teensy. Do you know for sure that the STM32 can operate dual-SPI reliably in slave mode at these speeds? The USB transfer rate (and therefore onboard RAM) shouldn't be a bottleneck if the MC receives the byte stream from 2 dedicated SPI modules. I just haven't seen any that can operate as a slave at that speed.

On a Teensy 4.1, here’s how i did it: https://forum.pjrc.com/index.php?threads/spi-slave-for-t4-dma-spi-slave.73418/

So two hardware SPI ports, both in slave mode, that capture MISO and MOSI data in DMA buffers.

That looks very promising, but your test example operates only at 6-8MHz. Do you know what's the speed limit or could you test it?

Rpi Zero 2w has got 2 SPI you may utilize. Will do 50+MHz SPI clock sure, 512MB of dram, 4 core 1GHz cpu, and XXX GB of flash on the sdcard (it can write 10MB/sec sure).. $15 they say..

But will it do this SPI speed in slave mode?

[Novgorod]

It can set 24 MHz sampling rate, but you don't care about that if you are using external clock. As long as it can receive 33 MB stream, it will be fine. You are not getting protocol decode anyway, since there will be no clock. All you get is export in CSV or similar.

The software is not a problem - as long as there's a library for configuring a capture and initiating a bulk transfer to some local buffer (on the PC), I can work with it. It's very simple with FTDI devices and I guess Cypress should be similar.

Have you read my question about "using an external clock"? Do you have more infos about that? Is it just replacing the onboard clock (crystal + divider) with an external source? Then surely it won't work if you simply wire the SPI clock from the device under test to it, because it's not a constant clock, and how would you configure the dividers for all the other components anyway (like USB)? Does it have an asynchronous mode where it's still using its own clock for USB etc. and the external clock only for data sampling, and what's the speed limit for that?

[ataradov]

Have you read my question about "using an external clock"? Do you have more infos about that?

FX2LP FIFO interface can use internal clock or external clock from the IFCLK pin. This is configured via IFCONFIG register.

One issue that comes to mind here is that it might actually expect continuous clock, not interrupted SPI clock. I think it might use that clock not just for the capture, but for the internal operation of the interface. The datasheet states that external clock needs to be at least 5 MHz. I don't know if that would actually matter, it still might work, assuming there is continuous stream, but it would require some experimentation,

[iMo]

..

Rpi Zero 2w has got 2 SPI you may utilize. Will do 50+MHz SPI clock sure, 512MB of dram, 4 core 1GHz cpu, and XXX GB of flash on the sdcard (it can write 10MB/sec sure).. $15 they say..

But will it do this SPI speed in slave mode?

The mcus can do slave usually at half of their max master SPI clock, afaik. The master SPI clock of that chip is at least 83MHz (I've been using that with my sdcard), but I bet it is much more..
And you may stream it upstream via usb2.0 as well..

PS: max SPI clock should be core_freq/2 = 200MHz (its stock core_freq=400MHz, or higher)..
You may set the cpu_freq, core_freq, dram_freq, gpu_freq "independently", incl "overclocking" them a bit (with a heatsink as it will throttle down at 80C)..

[Nominal Animal]

Or a CYUSB3KIT-003 and this firmware:
https://community.infineon.com/t5/Knowledge-Base-Articles/EZ-USB-FX3-Explorer-kit-as-16-channel-100-MHz-logic-analyzer-with-sigrok/ta-p/283993

Good suggestion.  The board itself is only about 45€ at Mouser, and the above link shows how to use it with Sigrok Pulseview, i.e. as a complete kit.  Basically no DIY needed for 100 Mbit/s per channel.

For real big use case boost, add TI ISO7740 digital isolators (4 in, 0 out), which provides full isolation, can handle 100 Mbit/s, and can do 2.25V to 5V logic level shifting too.  This way, there is no risk of ground loops (current loops due to slightly different GND potentials).  Only needs two 100nF=0.1µF capacitors, and costs 2€ in singles at Mouser.  Better yet, make it an add-on board at JLCPCB, which sells ISO7740DWR (SOIC-16) for their assembly service for $1.29 apiece.  In addition to the signals being sniffed, you'll also need the corresponding VCC plus GND for the isolators.

On a Teensy 4.1, here’s how i did it: https://forum.pjrc.com/index.php?threads/spi-slave-for-t4-dma-spi-slave.73418/

Did you ever check how high you can go with a slave SCK?

As the PLL used for the SPI peripherals runs at 720 MHz, I'd kinda expect something like 100 MHz to be the maximum.  Yes, yes, I know: I could just test it myself, using one Teensy as the master, and another as the slave, and just check it with e.g. a 64k sequence of Xorshift64* PRNG for binary verification.  I'm too lazy and not that invested in this, only curious; I've only needed to sniff at relatively low-frequency (few MHz) bus signals thus far.  For USB, I've relied on Wireshark on the host, not electrical sniffing.

[S. Petrukhin]

I didn't talk about the logic analyzer on board the MCU.
I was talking about hardware receiving the bit stream and reading the byte stream at the SPI port level. Of course, using external CLK.
What's the point of not using external CLK and reading the exchange while hanging in the air on an local CLK?

I can definitely say that the SPI STM32F407 is stable at SYSCLK/2 = 84 MHz, this is in ocumentation, and PIN allows 100MHz.
I use it to control w5500. I don't see any reason why he won't be able to work as a slave on the same frequency.

Next, I suggested using Ethernet.  You can create 4 UDP streams or MAC level stream (2x2pinpong) in time to send to the PC.
MAC is less costly in overhead, but at this level in a modern OS you need to have the skill to work, but UDP is trivial.
It will obviously be faster than USB 2.0.

After all, you can use two separate MCUs for MOSI and MOSO. 

But I think the Rpi, proposed iMo, will really solve the problem as simply as possible.

[std]

I need to monitor/record rather fast SPI bus activity at ~33MHz over extended time periods (tens of seconds) to a PC for analysis and I'm wondering what's the most efficient/cheapest/easiest method to do this.
Of course that's a trivial task for a logic analyzer, but at these bus speeds it requires at least 200Msps sampling rate with a very big onboard memory or a USB 3 interface for real-time streaming to the PC, which is too costly.

I believe that all other ways in practice will be too costly and long. TL/DR;  You need middle-class amateur logic analyzer, better on USB 3.0 bus.  You already formulated necessary requirements (real needed x5..x6 bandwith 200Mhz).  This amateur devices meet your specs are inexpensive (~$138..$149)
https://www.qdkingst.com/en
https://www.dreamsourcelab.com/product/dslogic-series/

[iMo]

Rpi as the SPI slave - doublecheck whether there is a working driver available actually.. 

[Novgorod]

One issue that comes to mind here is that it might actually expect continuous clock, not interrupted SPI clock.

That's what I thought. Usually these external clock options are meant for phase-lock to other devices and require a constant (within some percent) and known frequency which you have to tell the hardware somehow (jumpers or firmware config) in order to set up the correct dividers. Otherwise, this feature would be heavily advertised as asynchronous FIFO (which FTDI does, though I think it's fake because the transfer rate is reduced by 6x compared to synchronous mode).

The mcus can do slave usually at half of their max master SPI clock, afaik. The master SPI clock of that chip is at least 83MHz (I've been using that with my sdcard), but I bet it is much more..
And you may stream it upstream via usb2.0 as well..

PS: max SPI clock should be core_freq/2 = 200MHz (its stock core_freq=400MHz, or higher)..
You may set the cpu_freq, core_freq, dram_freq, gpu_freq "independently", incl "overclocking" them a bit (with a heatsink as it will throttle down at 80C)..

That sounds promising, I will look into it. It's just that I haven't found a fast (enough) SPI slave/sniffer implementation on the usual popular cheap MCs. A full fat Pi (zero) should be definitely more than fast enough hardware-wise if the peripherial interface isn't somehow excessively handicapped (which it doesn't seem to be?)..

The board itself is only about 45€ at Mouser, and the above link shows how to use it with Sigrok Pulseview, i.e. as a complete kit.  Basically no DIY needed for 100 Mbit/s per channel.

Yes, that's great value for money, but also (at least) a factor of 2 too slow for a general-purpose logic analyzer in this situation, as I explained earlier.

I can definitely say that the SPI STM32F407 is stable at SYSCLK/2 = 84 MHz, this is in ocumentation, and PIN allows 100MHz.
I use it to control w5500. I don't see any reason why he won't be able to work as a slave on the same frequency.

It's because in slave mode it has to lock onto an external data clock which can be completely different from the internal system clock (and non-constant), so it requires some intermediate buffering to acquire the received data at "non-deterministic" times and process it with the internal clock. Depending on the implementation it can vary from "a bit slower" (someone said half the max. master SPI speed) to a lot slower (22MHz on a Teensy, linked above). I'm no expert by any means, so I don't know if it's a hardware limitation or if it can be made "fast" with clever firmware tricks.

Next, I suggested using Ethernet.  You can create 4 UDP streams or MAC level stream (2x2pinpong) in time to send to the PC.
MAC is less costly in overhead, but at this level in a modern OS you need to have the skill to work, but UDP is trivial.
It will obviously be faster than USB 2.0.

The maximum efficiently packed (uncompressed) data rate is "only" 66Mbps, so USB 2.0 should be plenty.

[Novgorod]

This amateur devices meet your specs are inexpensive (~$138..$149)
https://www.qdkingst.com/en
https://www.dreamsourcelab.com/product/dslogic-series/

The onboard buffer size is too small to be useful, and PC streaming over USB 2.0 is always limited to 3 channels @ 100Msps. the FX3 board costs significantly less and supports 16 channels @ 100Msps. But 100Msps is just not fast enough for this application.

[ataradov]

That's what I thought. Usually these external clock options are meant for phase-lock

You are confusing operational clock and FIFO interface clock. That clock can be variable anywhere from 5-48 MHz. And I'm not sure that it actually can't be stopped. The only issue that may happens is that the last data would not be sent, but since it is a stream, there is no "last data".

And this feature is advertised, since it is literally the only useful feature of that product.

I would not use it that way in a commercial product, but for a quick hack to get the job done, I see no issues.

[Nominal Animal]

You can create 4 UDP streams or MAC level stream (2x2pinpong) in time to send to the PC.
MAC is less costly in overhead, but at this level in a modern OS you need to have the skill to work, but UDP is trivial.
It will obviously be faster than USB 2.0.

No, w5500 maxes out at 100 Mbit/s (100Base-T), whereas USB 2.0 High Speed maxes out at 480 Mbit/s.  What I listed w/Teensy 4.x is practical sustained bandwidth with no losses when the MCU generates the data, not theoretical maximum bandwidth or a special case.

USB 3 and gigabit ethernet are a separate matter.  The nearby GigE Micros thread discusses some of those options.

With CYUSB3, the only problem for me is that the GPIF II Designer is not available for Linux, and reportedly does not run (well enough to be used) under Wine either.  I have no Windows licenses, so a VM is pretty much out as well.  And I don't want to misuse MS licenses (say, on their development VM images), so CYUSB3's GPIF II is basically out for me.

After all, you can use two separate MCUs for MOSI and MOSO. 

Makes combining the streams into a coherent stream very difficult.

Yes, [CYUSB3]'s great value for money, but also (at least) a factor of 2 too slow for a general-purpose logic analyzer in this situation, as I explained earlier.

If you have Windows, you could register at Infineon, download the EZ-USB FX3 SDK, and run the GPIF II Designer to see if you can use the sniffed SCK as a trigger for latching the two inputs; probably will have to latch a full 8-bit byte, but that's okay; there's plenty of bandwidth.  I have registered (I have an EZ-USB FX2), but not being able to run the GPIF II Designer as it is Windows-only, I cannot verify this.  Based on the general descriptions, I do believe it is possible; GPIF and GPIF II are basically limited state machines with several control inputs, it's just that Cypress/Infineon for some reason really don't want to tell users how it is controlled, and only provide a Windows GUI tool to configure it.  It even uses a separate 8k RAM for its state machine description, I understand.

[mianos]

> The onboard buffer size is too small to be useful, and PC streaming over USB 2.0 is always limited to 3 channels @ 100Msps. the FX3 board costs significantly less and supports 16 channels @ 100Msps. But 100Msps is just not fast enough for this application.

Discounting the cost of the Dreamsourcelab, it depends on the data. The fgpa has RLE compression that occurs before the buffer is sent to the host.

It would interesting to try it.

I have one but I have never had anything that would produce data at 44Mhz. (Mine is used for SPI most of the time but not that fast).

[DavidAlfa]

Why not? Sampling is 3x faster than 33MHz, should capture it nicely.

[S. Petrukhin]

I can definitely say that the SPI STM32F407 is stable at SYSCLK/2 = 84 MHz, this is in ocumentation, and PIN allows 100MHz.
I use it to control w5500. I don't see any reason why he won't be able to work as a slave on the same frequency.

It's because in slave mode it has to lock onto an external data clock which can be completely different from the internal system clock (and non-constant), so it requires some intermediate buffering to acquire the received data at "non-deterministic" times and process it with the internal clock. Depending on the implementation it can vary from "a bit slower" (someone said half the max. master SPI speed) to a lot slower (22MHz on a Teensy, linked above). I'm no expert by any means, so I don't know if it's a hardware limitation or if it can be made "fast" with clever firmware tricks.

The SPI port buffer in the MCU, as far as I understand, is a regular shift register. His reaction to CLK and MISO is in no way controlled by the internal generator.

[iMo]

You are crossing the domain - from the external spi clock (extrnal master) to the internal MCU_slave system clock(s).. As I wrote above the max slave's SPI clock (coming from outside) is usually half (or even 1/4) of its spi subsystem clock..
Like when an mcu has max 100MHz spi clock as the master, as slave it is 50 or 25MHz..

[S. Petrukhin]

Interesting information...
Probably, the intersection does not occur at the input of the bitstream, but on the MCU's ability to detect the received byte and pick up  it from the buffer during the CLK front...

It became interesting for me try to connect 2 STM32 at 84 MHz. 

[iMo]

While crossing the clock domains you have to "sync" the asynchronous clocks somehow - either syncing as described above (1/(2..N) clocks), or with hw FIFOs around the SPI shift registers (not sure the SPI slaves use it) or with special approaches like Gray codes..

[S. Petrukhin]

I don't quite understand what you're talking about.
Not see any similar restrictions in the STM32 documentation or did not notice them.
And don't know how the MCU's MISO electronics actually work.
Personally, I would put the usual shift register and hyphenation scheme at the end of receiving a word. Here and further difficulties may arise with the movement of the byte.

But, since difficulties do not arise when descending a byte for transmission, I assume that it should work.
Although, I saw fading between bytes during transmission, but other tasks were being performed at that time, obviously.

There is a reason to explore.
I'll try to check it out. 

[Jeroen3]

I need to monitor/record rather fast SPI bus activity at ~33MHz over extended time periods (tens of seconds) to a PC for analysis and I'm wondering what's the most efficient/cheapest/easiest method to do this. Of course that's a trivial task for a logic analyzer, but at these bus speeds it requires at least 200Msps sampling rate with a very big onboard memory or a USB 3 interface for real-time streaming to the PC, which is too costly. I know there are certain attempts to use cheap microcontrollers as logic analyzers, but they have too little onboard memory, so even if they're moderately fast (ESP32 or Pico?), the USB interface is a bottleneck.

You are explaining what the Saleae Logic Pro 8 does...?
It can to 500 MS/s for 4 digital channels. Plenty for what you need. And it streams to your pc's memory.

So we now know 900 EUR is "too costly", this means any solution needs to be less then a day of development work. Or it will be more expensive.

[iMo]

I don't quite understand what you're talking about.
..
I'll try to check it out. 

The mcus usually have so called "synchronizers" at its inputs - two or three flip-flops clocked by their internal clock.
Also the moment the slave's SPI shift register is full is asynchronous with its internal clcok - the slave has to be in sync as well.
Not sure it applies for SPI and the stm32 - you have to doublecheck.

https://en.wikipedia.org/wiki/Metastability_%28electronics%29

[Novgorod]

If you have Windows, you could register at Infineon, download the EZ-USB FX3 SDK, and run the GPIF II Designer to see if you can use the sniffed SCK as a trigger for latching the two inputs; probably will have to latch a full 8-bit byte, but that's okay; there's plenty of bandwidth.  I have registered (I have an EZ-USB FX2), but not being able to run the GPIF II Designer as it is Windows-only, I cannot verify this.  Based on the general descriptions, I do believe it is possible; GPIF and GPIF II are basically limited state machines with several control inputs, it's just that Cypress/Infineon for some reason really don't want to tell users how it is controlled, and only provide a Windows GUI tool to configure it.  It even uses a separate 8k RAM for its state machine description, I understand.

This would be actually a great solution if it can keep up with 33MHz in asynchronous FIFO mode - something like the FT232H but much faster, so I definitely want to look into that. I'm just a bit cautious because I know the GPIF II is limited to 100MHz when using an internal clock and there's some factor of performance loss in asynchronous mode (factor of 6 for the FT232H, and someone mentioned a factor of 2-4 for different MCs). So it would only work in the most optimistic case when there's only a factor of 2 slow-down. I've looked a bit through the FX3 docs, but couldn't find any performance figures for the async FIFO mode...

Discounting the cost of the Dreamsourcelab, it depends on the data. The fgpa has RLE compression that occurs before the buffer is sent to the host.

It would interesting to try it.

That's a big maybe.. The compression is a good point, but the SPI clock is relatively "high entropy" for simple RLE compression, so essentially it only removes dead space between data bursts. At 400Msps you'll get at most 6 same-level samples on a clock pulse, how much can you compress that with RLE? A factor of 2 at best? The 256MBit memory can record 0.2s for 3 channels, so even with the most optimistic compression, it can be stretched only to 1s or so.

Why not? Sampling is 3x faster than 33MHz, should capture it nicely.

Because Nyquist only applies for analog sampling and a rectangular waveform has much higher harmonic content than its fundamental frequency. If it's perfectly sharp, you're guaranteed to have at least one sample on each half-cycle (15ns width) at 100Msps sampling rate (10ns samples), then you "only" have to deal with enormous timing jitter (~50%). If you take into account some finite rise and fall times and put the analog threshold at 50% amplitude, there's a chance to miss pulses completely if the threshold levels happen to be less than 10ns apart.

You are explaining what the Saleae Logic Pro 8 does...? [...] So we now know 900 EUR is "too costly", this means any solution needs to be less then a day of development work. Or it will be more expensive.

It's a hobby project and I have no other foreseeable private use for a professional tool like that, so it would be completely unreasonable to invest this kind of money if I can get the same result with a 15-45$ tinker board. I also don't pay myself a salary for my free time.

[Nominal Animal]

I'm just a bit cautious because I know the GPIF II is limited to 100MHz when using an internal clock and there's some factor of performance loss in asynchronous mode (factor of 6 for the FT232H, and someone mentioned a factor of 2-4 for different MCs).

The only way to find out is to use GPIF II Designer, because it is the only "documentation" available for it.  As it runs only in Windows, not even under Wine, I cannot.  That's why I asked.

[langwadt]

I did something like that nearly +20 years ago with an FPGA and a BM245 ftdi fifo IC.  Each spi frame was sent with time stamp to a PC, the 245 is only ~1MB/s  but it was fast enough

With it USB2 should be enough for 33MHz spi

If you don't mind some verilog, maybe a Gowin TangNano 9k? I see they have a soft USB2 HS phy so a $17 board and a couple of resistors

[std]

The onboard buffer size is too small to be useful, and PC streaming over USB 2.0 is always limited to 3 channels @ 100Msps. the FX3 board costs significantly less and supports 16 channels @ 100Msps. But 100Msps is just not fast enough for this application.

I believe that the highest quality answer follows only a complete understanding of the person’s tasks, goals and intentions. What is your true goal? Do you need to record and (for some reason) play back signals? Or do you need to perform protocol analysis? Or maybe the question was asked out of sporting interest? How much time is allocated to complete the task? I used the EZ-USB FX2LP along with Sigrok for fun to make the cheapest possible LA ($5-$6). If you want to build an extremely cheap but fast LA, then undoubtedly the easiest way is to take a cheap FPGA board and integrate it into Sigrok. Like guys here https://www.eevblog.com/forum/microcontrollers/please-help-me-to-select-an-usb3-fpga-board-for-my-logic-analyzer-project/ LA based on the FPGA that I gave you above compresses static signal and have triggers for events. In general, the analysis work does not actually represent a long-term recording of the signal, it is an iterative and dynamic process, and the software shows the signals dynamically like an oscilloscope, and in general, 99% of success is in the software. At the end of 2019, I spent several months on tasks that you roughly want, the result was a discarded mountain of code, several boards, plenty of cables, connectors and wasted time. (Instead of leaving a useful and important equipment in the laboratory). Already when I went through all this, I purchased DSLogic Pro and realized what a fool I was for not knowing about the capabilities of its software. By the way, DSLogic Pro no longer suits me; now in 2024 there are often boards with quad or octa SPI at frequencies of 80 MHz and higher, as well as other fast buses.