SoC as alternative to STM32H7

[arijav]

Hello,

I am currently developing an measurement and control board based on the STM32H743 microcontroller. I am driving several ADC's each sending 8 channels to the uC as well as several ADC's reading thermocouples and sending the data over ethernet. The board controls as well digital outputs and contains other secondary functions.

While the STM32H7 series is a very powerful microcontroller, I face some limitations:
1- Ethernet is just 10/100, so the data I can send is limited. This is a limitation I can not overcome with more powerful dual core STM32H7
2- High speed acquisitions (each ADC channels could go up to 144KSPS) in interrupt time can be handled but in limited numbers

For 16 channels + 18 thermocouples I manage somehow to get it working, but for a bigger setup the uC come to its limits.

I am therefore looking for alternatives. I have seen the Xilinx Zynq 7000 series (the 7020) and they seem attractive for the application. The uC is powerful and they have an FPGA which could handle reading the data from the ADC's and putting it in a easy way for the uC to send it over Ethernet (gigabit).

I wanted to ask you your opinion of using the Zynq 7000 series for this application as well as potential alternatives that you have in mind.

Thanks

[tom66]

Zynq is a good processor, but if you want to buy it on volume then prepare for 12-18 month lead times or to buy on greymarket (though not as bad as Spartan 6)

If power consumption is your goal then you will struggle with a Z7000.  The lowest we have achieved is around 0.8-0.9W with both CPUs asleep.  With any work done on the CPUs or if the DDR is busy then power is going up to 1.2-1.5W.

The Zynq only has 256KB of usable OCM RAM (you cannot use the L2 as a RAM unlike some CPUs), if you need any more then you will need DDR.  Note that the program needs to fit into this as well (no onboard flash to hold application), so DDR is pretty much essential for all but the simplest designs.  The FPGA does have SRAM arrays to hold state once programmed, but still, it's going to be tight without DDR.  This also means you need a boot device of some kind, we use SD cards but it can boot from QSPI and NAND as well, and  technically eMMC but this has poor official support.

Another option is to couple a regular-old FPGA with a parallel bus interface on some CPUs, e.g. one used for memory access.  The FPGA can emulate this interface, so your CPU just reads/writes at addresses.  If you get this right, then the CPU DMA can also be used so you can do a fast copy into main memory or you can work on the data directly as a memory mapped interface.

If power and supply chain are not major issues for you,  then the Zynq is a good chip and I do find it quite pleasant to use.  But, if you have not designed with an FPGA before, the learning curve can be pretty steep.

You probably don't need a 7020 for the application you've described (and it's like 100 euros a chip, so quite expensive.)  A 7007S is probably enough, providing you don't need to do much DSP with what  you capture.

[arijav]

Thanks for the inputs. Regarding the application:

- Volume/Part availability: It is for now very low volume, max 20 units/year. I want to replace with inhouse development NI CompactRIO Systems (some of the old ones use the Zynq 7000 series) that we are using in test benches. Do you know where I could get a few 7007S parts from now?
- High Power consumption: 1.2 to 1.5W is not an issue, the STM32H743 is a power beast as well
- 7020: I've thought about it because it is on stock in JLCPCB. 50€ difference to the 7007S is not a big deal for me as it is for low volumes and the NI CompactRIO Systems I want to replace cost 15K€ each...

I've thought about the alternative you've mentioned of a simple FPGA to the QuadSPI Iface of the STM32H7, but I still get the issue of the low Ethernet speed (no gigabit).

[tom66]

You might fare better using a MYiR Z-Turn or Z-Turn Lite board on a 120 pin Samtec header if volume/space is not a serious issue.  You can get these on relatively short lead times and they have Ethernet and USB on them so no need to lay out a board. 

e.g. https://www.mouser.co.uk/ProductDetail/MYIR/MYS-7Z010-C-S?qs=lc2O%252BfHJPVY%252Bw34Et00SDQ%3D%3D

Unlike some SoCs the Zynq does not have internal phys for the Ethernet or USB so you will need those chips as well on a design.

7007S: probably from a broker, like the dreaded Win-Source...  No stock at distributors...  But if cost is no problem then use the 7020, you will definitely not run out of logic space there!

Be aware if you're planning to use Zynq's XADC it's 2 x 1Msps multiplexed so you'd probably need external ADCs for what you're doing.

[langwadt]

Thanks for the inputs. Regarding the application:

- Volume/Part availability: It is for now very low volume, max 20 units/year. I want to replace with inhouse development NI CompactRIO Systems (some of the old ones use the Zynq 7000 series) that we are using in test benches. Do you know where I could get a few 7007S parts from now?
- High Power consumption: 1.2 to 1.5W is not an issue, the STM32H743 is a power beast as well
- 7020: I've thought about it because it is on stock in JLCPCB. 50€ difference to the 7007S is not a big deal for me as it is for low volumes and the NI CompactRIO Systems I want to replace cost 15K€ each...

I've thought about the alternative you've mentioned of a simple FPGA to the QuadSPI Iface of the STM32H7, but I still get the issue of the low Ethernet speed (no gigabit).

for such low volumes just buy a board, it has everything for less than you can buy it and is know to work

https://digilent.com/shop/cora-z7-zynq-7000-single-core-and-dual-core-options-for-arm-fpga-soc-development/
https://www.avnet.com/wps/portal/us/products/avnet-boards/avnet-board-families/microzed/
http://www.myirtech.com/list.asp?id=565

[betocool]

I'm curious...

Thermocouples and Gigabit? I have an H743 sending real time audio data over Ethernet at 96KHz sampling rate without breaking a sweat.

How fast are you sampling your thermocouples? Is it really necessary? Think about that.

A few years ago at work we managed 40 ADC channels at 12 KHz/16bit over Ethernet with an F7. That's ~8Mbit. But! We had a 512MBit SDRAM attached to it to buffer the data. Still, unless there were network issues, it managed. That one was under heavy load.

Disclaimer: I'm genuinely curious, and not trying to sound smug or smarter. Just pointing out things than can be achieved on an H7.

Cheers,

Alberto

[laugensalm]

The Zynq Stuff is far too heavy weight for what you're trying to achieve. Programming those in bare metal is quite a bit of a pain, and the SoC interfacing involves quite some overhead, if you're doing it the 'X' way. But they have their point, you might also want to check the 'Red Pitaya' approach.
I've got some reference solutions based on ECP5 and the good old Spartan6 using a 100MBit DMA-driven Soft-EMAC. For most analog data you can use lossless (or quantized/lossy) DPCM methods and obtain compression factors < 1/10. That should by far apply to thermocouple data, unless it's very noisy and when you'd want to keep that noise. I've based pretty much everything on RTP/UDP which gives a sufficiently good timestamp for most applications, when you have a good 90 kHz source. But there's a bit of a shopping list of things you need to have, like a proper scatter-gather based DMA and a UDP stack that plays well with it (unlike lwip). The Soft-CPU is then only used for setup, networking/command state and interrupt handling.
And I heavily assume you're using UDP with the usual packet drop reality and corresponding handling included, so your DPCM-unit has to packetize, of course.

One advantage of that is also that your network setup  scales well, one good GigE hub will at least allow ~10 scope units to transmit smoothly. It's a bit of an investment to do this all on an FPGA where you could just make a classic ARM do the job, but it can pay off quickly when debugging to the bits is required and when you need full potential of customization.

[arijav]

I'm curious...

Thermocouples and Gigabit? I have an H743 sending real time audio data over Ethernet at 96KHz sampling rate without breaking a sweat.

How fast are you sampling your thermocouples? Is it really necessary? Think about that.

A few years ago at work we managed 40 ADC channels at 12 KHz/16bit over Ethernet with an F7. That's ~8Mbit. But! We had a 512MBit SDRAM attached to it to buffer the data. Still, unless there were network issues, it managed. That one was under heavy load.

Disclaimer: I'm genuinely curious, and not trying to sound smug or smarter. Just pointing out things than can be achieved on an H7.

Cheers,

Alberto

Hello Alberto,

thanks for your input. I can explain a little bit better my application with the H7. I am fully open for suggestions to improve it:

Currently the STM32H7 is:

1- Acquiring via 2xSAI buses 16x ADC 24bit channels comming from 2 external ADCs over isolation barriers. These are not thermocouples but high speed analog acquisitions. I manage with Raw LWIP and w/o OS 115KSPS per channel (when sending just these ADC signals over ETH) but unfortunately the ST implementation of LWIP is crap and it hangs after 10 hours. With FreeRTOS I manage with the same setup and an no LWIP but FreeRTOS+TCP 85KSPS per channel. I might improve this but I have to check that I move the buffer to a 512K RAM block. The HW that I have prepared is actually meant for 32 ADC 24 bit channels (2 additional ADCs can be populated). By the way, the ADC itself would be capable of delivering 144KSPS per channel.

2- Acquiring 18x 24bit low speed measurements over isolation barriers (16 Thermocouples + 2CJCs). Here I do not acquire more than 100-150SPS per channel. The acquisition is over SPI.

3- Due to the requirements of the application (full sync. acquisition), I have unfortunately to send both high speed signals as well as low speed signals in the same packet. This means, even if I acquire the Thermocouples at very low speed they have to be consolidated together with the high speed signals and send at the same rate over ethernet. I have to send as well at high rate over ethernet status signals for 16 digital outputs and additional diagnostic data. If I send just the high speed signals in the packets + diagnostic and output status I get the above mentioned 85KSPS/channel (or 115KSPS/channel) for the fast signals, if I send additionally the 18 low speed signals at the same rate together with the rest I get arround 49KSPS/channel for the fast ones.

4- For absolutely all I use DMA and Zero copy for Ethernet. @laugensalm, I am using TCP, as it is important for the application to not miss any data.

My bottleneck is both the Ethernet speed, as well as the MCU load. I would like also to extend the number of Acquisition signals from 16 to 100 in the future, so I do not really see the STM32H7 as capable enough for the application.

[betocool]

OK, I understand a bit better now, thanks!

I did not do the calcs on paper, but I 'guesstimate' that with an H7 you should manage up to 40Mbit or so. If that is enough, my approach is FreeRTOS + LWIP, where I store incoming data into a StreamBuffer (FreeRTOS' FIFO) and read out with the TCP Sender function. Unfortunately from in-buffer to out-buffer there is data copy involved. That has allowed me to stream 96KHz audio without glitches. You will need to pre-define your data structure to send all fast and slow data in the same stream, plus add some checksums for good measure.

My other approach would be, if you need GBit, to do the jump to a Linux system, with the ADC data being sampled in the FPGA, and stored into memory via DMA or any other methods FPGA's use these days for data transfer. Example, Intel/Altera has a Scatter Gather DMA which lets you grab data from a source, specify the RAM write address, and then the blocks take care of the rest pretty much. I don't know about Xilinx, haven't explored it. I would be surprised if they didn't!

Cyclone V offers dual ARM cores with FPGA fabric, but I would not do bare metal on them, let the OS take care of everything.

If you have never built an embedded Linux system, it is a relatively steep curve. Of course, manufacturers offer SD card images ready, but when you want to customize things, that's when it gets tricky. Prepare to spend one or two weeks depending on your skills getting your new image from scratch. And the time it takes to compile.

If price is a concern, you can't beat STM32 H7 dev kit for a few bucks. But you're limited to lower bandwidths.

Terasic / Digilent and other offer a range of boards with ARM hard cores + FPGA, but prices are usually in the 200+ USD or so region I believe. More for more powerful devices.

Not being too helpful, sorry!, but I'd be interested to see what you do next. Data acquisition over Ethernet is a subject I'm very interested in and have looked at in the past one way or the other.

Cheers,

Alberto

[lty1993]

You can always try some high-performance MCU, such as TI AM2434 and NXP IMXRT1170. These MCU comes with Gigabit Ethernet and multiple processing cores.

Zynq definitely works in this situation, but it is much more complex regarding hardware and software design.

[SiliconWizard]

I would certainly NOT consider a Zynq unless I needed the FPGA part of it as well. If you only need the ARM SoC in it, that's a huge waste of resources and energy.
Now given the OP's application, the FPGA fabric could certainly be put to use, but that would require a significant redesign. So, yeah.

[arijav]

Dear all,

first thanks a lot for your valuable inputs!. I am learning a lot with your comments and discovering possibilities I did not thought about.

@Alberto: I would be happy to keep in touch with you for the further development. Data Acquisition over Ethernet will be my main dev. topic for the next months, maybe years. Regarding FreeRTOS+LWIP, I was wondering how did you manage to get it running. I did try w/o OS and Raw LWIP and it works for a while but the communication is not reliable after some hours. There are many problems reported since years over buffer issues where head/tail come together and the communication stops and more topics and even with the new rework of the ETH stack problems are still there. FreeRTOS+TCP works on the other hand like charm for me. By the way, for the current revision of my HW I will stick to the STM32H743, for the next one I will go dual core (but first I've to find some STM32H745 on stock somewhere, it might take months) and for the third one let's seen, maybe Xilinx or more powerful ARM MCUs.

@lty1993: Thanks for your input about the high-performance MCU from TI and NXP. I might give them a try as an alternative to the Xilinx SoC.

@SiliconWizard: I get your point and I will surely explore as well ARM SoCs. For the Xilinx I though about the FPGA part for acquisition over the buses (SAI/SPI) the DSP and building up the ETH payloads and the ARM for sending the TCP packets fast over gigabit Ethernet.

[kmike]

Do you really need the ARM core for your application?

You could use a normal FPGA without a hard core and do the TCP stuff in a HDL language of your choice. For Xilinx FPGAs you could use a core from fpga-cores.com, I have tried their FC1001, works perfectly.

br,
mike

[arijav]

Do you really need the ARM core for your application?

You could use a normal FPGA without a hard core and do the TCP stuff in a HDL language of your choice. For Xilinx FPGAs you could use a core from fpga-cores.com, I have tried their FC1001, works perfectly.

br,
mike

Actually this would be an alternative but maybe with a steeper learning curve, as the whole staff I am doing in the ARM apart of the acquisitions would need to be translated to an HDL. For me it is however difficult as well to guesstimate which size of FPGA I would need. Would a small FPGA (let's say a Gowin GW1NR-9) be enough or do I need something in the range of 25K to 50K LUTs?

[laugensalm]

I did try w/o OS and Raw LWIP and it works for a while but the communication is not reliable after some hours. There are many problems reported since years over buffer issues where head/tail come together and the communication stops and more topics and even with the new rework of the ETH stack problems are still there.

There are some serious architectural flaws in lwip with respect to its buffer queue, there are forks that work ok, some only on some CPU architectures, etc. so this is not exactly robust for a 52/7/24 streaming requirement. And I read you're using TCP. Number one rule: Go with UDP and a proper protocol. TCP will unevitably lead to bandwidth problems, especially with the packetizing performance of lwip which does not use up the GigE potential. Not a problem when you've got a controlled peer to peer connection with no packet interference, but then you might just as well use UDP. I'm running approx. 1000 packets per second with a bit of redundancy for weeks without packet loss, FPGA is a Spartan6 LX-9.
Problem with TCP is scalability and timeouts, which can cause buffers to run full after, say, 25 days. The question always is: Can you skip over a packet after 25 days or would you rather lose the stream for a few seconds and need restarting?

Doing IP stacks in HDL is a pretty ineffective idea, by the way. Just make sure you fit in a simple Soft-CPU and use a DMA core with autobuffer descriptors. That way you can directly stream from a camera port, for example, like on the STM32 (w/o CPU overhead) and still get the full bandwidth.

[tom66]

Building Ethernet payloads in an FPGA, whilst completely possible, is not going to be as trivial as doing it in a CPU.

Have you written Verilog before - it is not the same as writing C.   It sounds like you have relatively little FPGA experience. 

[arijav]

Building Ethernet payloads in an FPGA, whilst completely possible, is not going to be as trivial as doing it in a CPU.

Have you written Verilog before - it is not the same as writing C.   It sounds like you have relatively little FPGA experience.

No experience with FPGA. This is why my idea was to limit its part to the acquisition and DSP and for the rest (including TCP) using the ARM.

[kmike]

The FC1001 IP uses 1718 LUTs and 1323 FFs, so it happily fits in a smaller FPGA and does all packet handling itself. This is a UDP only IP, I have not used their TCP IP, so I cannot tell you the utilisation of that.

As laugensalm wrote, it would be much better to use UDP for your purpose

The utilisation really depends on what you plan to do inside the FPGA apart from acquisition and TCP transmission.
You can use a bigger FPGA for the development and then replace it with a smaller (pin compatible) one from the same family later.   

I have no experience with the Gowin products, but they have a free Triple Speed MAC IP, with modest resource requirements, so its also worth a look

[lty1993]

No experience with FPGA. This is why my idea was to limit its part to the acquisition and DSP and for the rest (including TCP) using the ARM.

Writing acquisition and DSP in Zynq will require you to write an efficient and working AXI bus slave (if even higher bandwidth is required, you have to implement AXI master and write an application-specific DMA module in FPGA and copy data directly to DDR memory through AXI_HP interface), which may not be an easy task if you have no experience in FPGA design.

[tom66]

You don't really need to write a true AXI-Full slave.

Depending on what you're doing you could use an AXI-Stream and interface it with a Xilinx DataMover block (which will require writing a command issuer) or an Xilinx DMA block (which you control from the CPU, it's pretty much a DataMover at heart.)

On the even easier end you can stuff the data into a BRAM and use an AXI BRAM instance to make this accessible to the CPU.  I haven't tried this but it then may be possible to interface this with the PS DMA blocks (ARM PL330 is what the Zynq has.)

However, all of this will still require some appreciation of the design flow for an FPGA & a willingness to set up ILAs and simulations to test an implementation, however simple.

[langwadt]

Building Ethernet payloads in an FPGA, whilst completely possible, is not going to be as trivial as doing it in a CPU.

Have you written Verilog before - it is not the same as writing C.   It sounds like you have relatively little FPGA experience.

No experience with FPGA. This is why my idea was to limit its part to the acquisition and DSP and for the rest (including TCP) using the ARM.

with no FPGA experience thats' going to be a hard learnign curve.

How about something like a i.MX RT1170 ? 

[betocool]

@arijav I last updated my repo with the Audio related project in January, I have not updated much (read, at all!) since January, but last I looked, it worked on an H743 Nucleo board.

The repo is a complete Stm32Cube IDE project, and unfortunately I don't have documentation either! So... let me see if this helps you or freaks you out...

https://github.com/betocool-prog/Audio_Analyser_FW_OS/blob/master/tasks/Src/controller.c

This module controls the ADC (PCM1804) and the DAC (PCM1793). It receives the data from the ADC (via I2S - DMA), and copies it into a StreamBuffer (line 138)

Code: [Select]

data_size = xStreamBufferSend(tcp_out_stream_buffer_h, (const void *)from_ptr, INPUT_BUF_SIZE_STEREO_SAMPLES * 4, 0);

There's a few more things about reading the complete transfer or half transfer DMA complete, and some synchronisation stuff going on, so that the ADC and DAC data are in sync.

The TCP bit is a separate Task, which reads the contents of the streambuffer (Starts in line 138). The micro is the server, the PC acts as client, which requests data. If a connection is accepted, whatever bytes are written into the streambuffer are sent over, so long as there are bytes, and as long as you can read a minimum amount. No point in sending small packages! Everything closes down in case of errors.

Code: [Select]

        if (controller_conn!= NULL)
{
/* Bind to port 80 (HTTP) with default IP address */
err = netconn_bind(controller_conn, NULL, 8888);

if (err == ERR_OK)
{
/* Put the connection into LISTEN state */
netconn_listen(controller_conn);
log_msg(LOG_INFO, "Listen OK: %s, line: %d", __FUNCTION__, __LINE__);
while(1)
{
/* accept any incoming TCP connection */
accept_err = netconn_accept(controller_conn, &newconn);
if(accept_err == ERR_OK)
{
log_msg(LOG_INFO, "Accept OK: %s, line: %d", __FUNCTION__, __LINE__);
start_ticks = 0;
stop_ticks = 0;
diff_ticks = 0;
got_start_ticks = false;
got_stop_ticks = false;
conn_accepted = true;

/* Write TCP data */
do_loop = true;
while(do_loop)
{

bytes_read = xStreamBufferReceive(tcp_out_stream_buffer_h, (void *)tcp_out_buffer, TCP_OUT_BUF_SIZE, 1);

if(bytes_read != 0)
{
err = netconn_write(newconn, tcp_out_buffer, bytes_read, NETCONN_COPY);
if(ERR_OK == err)
{
bytes_written += bytes_read;
}
else
{
do_loop = false;
log_msg(LOG_ERR, "Write NOK: %s, line: %d", __FUNCTION__, __LINE__);
}
}
}

conn_accepted = false;
netconn_close(newconn);
netconn_delete(newconn);
xStreamBufferReset(tcp_out_stream_buffer_h);
log_msg(LOG_INFO, "Bytes Written: %lu", bytes_written);
bytes_written = 0;
}
else
{
log_msg(LOG_ERR,"Accept NOK: %s, line: %d", __FUNCTION__, __LINE__);
}
}
}
else
{
log_msg(LOG_ERR, "Bind NOK: %s, line: %d", __FUNCTION__, __LINE__);
}
}

Somewhere in the LWIP config file I increased the size of the send buffers a fair bit, since the H7 has a lot, why not.

And yes, documentation for LWIP could be much better, given it's been around for 10 or 15 years. And yes, googling LWIP information is always going round and round to the same information. But I have not found a better alternative to be honest, and if you have some time you can get it working pretty well. It's that or writing a TCP stack myself... no thanks. And I don't mean that as disrespect to people who have, they are a level above me!

And I would stick to TCP, at high data rates UDP also becomes unreliable, and writing your own protocol over UDP is basically TCP... YMMV.

Cheers,

Alberto

[laugensalm]

And I would stick to TCP, at high data rates UDP also becomes unreliable, and writing your own protocol over UDP is basically TCP... YMMV.

I'd have a long reply to that, but it's a groundhog day topic, really.
You might also check this thread: https://www.eevblog.com/forum/fpga/fpga-high-speed-udp-stream-and-packets-processing-on-pc/
When your receiver drops packets (this being the most frequent cause in the field), the issues you'd have with UDP would seriously impact on TCP stability as well, and you'll never get the desired bandwidth, let aside any realtime guarantees.
You'd have to eliminate these issues anyhow - while TCP streaming stalls are harder to debug.
And no, you don't want to re-invent TCP. There are more efficient ways to issue packet resends or avoid those at all with some FEC (forward error correction) trickery. Depends all on your error/data scenario. Last way out is always to use EtherCAT or similar.

So, after all, there's some homework to be done, likely not for a FPGA SoC development, as this can take easily a few man years from scratch to complete. Not something that pays off with like 20 units p/a.

[betocool]

@laugensalm, I don't disagree with you!

Maybe I should have said that TCP has worked for me very well in private network scenarios, 192.168.x.x networks and such, no routers in between. And yes, the moment you start adding network issues, it all pretty much goes to hell very quickly at those speeds.

As an experiment we once added a Linux box as a router of sorts, where we added network delays and packet losses and what not between an STM32F7 and the receiver. Adding a few ms delay was enough to throw all overboard.

Cheers,

Alberto

[arijav]

@arijav I last updated my repo with the Audio related project in January, I have not updated much (read, at all!) since January, but last I looked, it worked on an H743 Nucleo board.

The repo is a complete Stm32Cube IDE project, and unfortunately I don't have documentation either! So... let me see if this helps you or freaks you out...

https://github.com/betocool-prog/Audio_Analyser_FW_OS/blob/master/tasks/Src/controller.c

This module controls the ADC (PCM1804) and the DAC (PCM1793). It receives the data from the ADC (via I2S - DMA), and copies it into a StreamBuffer (line 138)

Thanks a lot for your code Alberto. I am starting with FreeRTOS as I was using LWIP Raw API w/o OS before and I see already how can I speed things up thanks to your code..., I was not using the ISR task notification system!.

Another question, did you manage with LWIP to run continuous TCP traffic for long periods (>1 day)?. In my case the communication was hanging after 10 hours or so. I did not developed my own alternative stack, but changed LWIP with FreeRTOS+TCP. I really recommend it, it is super stable in the STM32H743:

https://www.freertos.org/FreeRTOS-Plus/FreeRTOS_Plus_TCP/index.html