I want to design my own STM32-based device, but I'm running into lots of issues

[SparkyTD]

Hi there,

I am pretty much a beginner when it comes to PCB design, but I would like to challenge myself by designing a custom STM32 based board that I will be using for my project. I have looked at many schematics for similar devices, mostly the Arduino Mega and Arduino Due, and I feel like I understand most things, and I would be comfortable giving it a go with my own custom schematic and PCB design. However, my biggest issue has been chip selection. I have a set of requirements for my project, and I need an STM32 that meets all of them. The problem is, that since I don't have the equipment or skill to solder such tiny SMD components, especially if I end up choosing a BGA chip, I will have to use an SMT service like JLCPCB to have my PCB manufactured, and the SMD components attached.

This means that I have to adjust my project to JLCPCB's stock availability, which changes daily, and by the looks of it, they don't have any powerful STM32s in stock. It would also be helpful if a development board would be available for the chip that I choose, so I can experiment on it, and learn the basics of the programming before I start designing my own PCB. This reduces the number of available parts even further.

Before I go further, here are the hard requirements of my project:

• At least 2 separate, user-accessible SPI interfaces
• Support for a microSD card reader
• Support for a WiFi module like the EMW3081
• 300kB+ SRAM (512 would be better, but I could probably make it work with 300+)
• 2-4MB Flash

I find it very difficult to find a chip that meets these requirements, and is in stock at JLCPCB. As a beginner, I don't think I should try doing stuff like external memory chips, because I'm not comfortable with more than 2 layers of PCB, and I have no idea how to do all the trace-length matching, etc...

Can anyone give me some advice on how to get started with this project? Or should I just give up and design the whole thing around a Raspberry Pi Zero W?

[vad]

Consider using external SPI or QSPI flash chip. As for the availability, Mouser has few STM32 models in QFN package in stock now. Otherwise back order parts now and be prepared to wait 52 weeks or so. If you need just several MCUs for prototyping or personal project, desoldering can also be an option.

[lucazader]

Due to the chip shortage right now its going to be very hard to nail down a chip months in advance from a supplier like JLCPCB.
Some PCB manufacturer and assembly places do allow you supply some parts (eg the hard to find MCU) yourself. And then they will source the rest (eg resistors etc).
At my company this is how we have been doing recent prototypes, via PCBway.

As far as selecting the correct chip:
For the RAM requirements you probably need to look at something in the F7 H7 or G4 range of microcontrollers from ST.
For Flash space. There are parts that do come in up to 2MB, however this is the largest that ST go to as far as built in flash.
If you need more than this, you will likely have to go with external QSPI flash, which as you have said is not really something that a beginner should really look to doing.

One thing you could look at doing is getting a pre made higher end MCU module that include most of what you need, and then making a carrier board for it.
You could look at getting a Teensy 4.x
Or something like the arduino pro portentia H7 lite connected
The portentia would work with the arm mbed platform so lots of good drivers out there for doing what you might need. It also include BLE and WiFi and a few other nice things.

[SparkyTD]

I've never heard of the Arduino Pro, but it seems like one of the coolest boards I've seen in a while, thank you for recommending it!

As for the PCB manufacturing, if I could get my hands on a chip from mouser/digikey/etc within reasonable time, do you think JLCPCB would accept it as a user supplied part? Or if not, could you recommend any other PCB manufacturers that do?

[vad]

For the RAM requirements you probably need to look at something in the F7 H7 or G4 range of microcontrollers from ST.

Also L4 and L4+

[vad]

As for the PCB manufacturing, if I could get my hands on a chip from mouser/digikey/etc within reasonable time, do you think JLCPCB would accept it as a user supplied part?

JLCPCB will not do that. Direct quote: “ Currently the components supplied by customer are not accepted by JLCPCB SMT. We provide parts you will need to have your PCB assembled. Please note components that are not in our library won’t be placed on your board.”

Why not learn soldering QFP or even QFN yourself?

[SparkyTD]

Why not learn soldering QFP or even QFN yourself?

I'm still just getting started with THT soldering, so I wouldn't trust myself with smaller SMD components, especially if it's a somewhat expensive MCU. Plus I would have to make an investment by buying a hot plate, a hot air station, and maybe even a new soldering station because my current one is very dodgy. I'm trying to keep the cost of this whole project as low as possible.

[vad]

QFP can be soldered with soldering iron.

[I wanted a rude username]

How core is the STM32 requirement for you? If you can compromise on that, JLCPCB carry a range of modules with Wi-Fi and up to 16 MiB of flash and 8 MiB of PSRAM: the ESP32. It's somewhat easier to get into than STM32 as well, and the price is right. The tradeoff is that if energy consumption is a concern, you need to carefully manage its various sleep modes.

[SparkyTD]

How core is the STM32 requirement for you? If you can compromise on that, JLCPCB carry a range of modules with Wi-Fi and up to 16 MiB of flash and 8 MiB of PSRAM: the ESP32. It's somewhat easier to get into than STM32 as well, and the price is right. The tradeoff is that if energy consumption is a concern, you need to carefully manage its various sleep modes.

The project that I'm working on is a "smart" electricity meter, and the current working prototype is actually based on the ESP32. The problem was that I had three SPI devices that I wanted to attach to the chip (A TFT display, a microSD card reader and an external ADC chip), but unfortunately out of the 4 SPI controllers that the ESP32 comes with, only two are user-accessible, the others are used by the Flash.

Since I the current prototype only supports two SPI peripherals, I had to give up on the SD card, and right now all the data is being stored on an Amazon server. My goal with this project is to make it completely independent from an internet connection, so all the data can be logged locally on the SD card, and accessed with an web browser through WiFi. I've also found that the ESP32 has trouble transmitting bigger websites (jQuery, amCharts, bootstrap, etc...) due to memory and bandwith constraints.

[Kjelt]

Can't you multiplex the display and sd card ?
You can buffer data and once in a while write it to the card.

[agehall]

Hang on a second - we are talking about a smart meter and the specs are reaching in to high performance STM32 ranges? This sounds more like a project for the AtTiny end of the spectrum rather than high performance STM32 processors.

I fully understand the desire to go with some sort of module like ESP32 and solve the rest of your needs with peripherals.

And as far as HTTP server issues with ESP32 modules goes, some tweaking might be in order once you fire it up, but done correctly, I have not experienced much of an issue with streaming data from it.

[brucehoult]

The project that I'm working on is a "smart" electricity meter, and the current working prototype is actually based on the ESP32. The problem was that I had three SPI devices that I wanted to attach to the chip (A TFT display, a microSD card reader and an external ADC chip), but unfortunately out of the 4 SPI controllers that the ESP32 comes with, only two are user-accessible, the others are used by the Flash.

How many of those actually need hardware SPI?

The display I guess, and maybe the ADC, depending on how often you do a sample. But I'd bet the bandwidth to the SD card would be low enough to bit-bang the SPI using GPIOs, and maybe the ADC also.

[Kleinstein]

A smart meter should have a smarter program and get away with a smaller µC.

The STM32 chips and especially the high performance ones are hard hit by the supply problems. So if just starting a design, I would not use them and look for something that is better available. Still hard to tell what is available in 5 years when a beginner finishes a 2 MB projects.

[DavidAlfa]

You're asking for a lot of memory, borderlining the transition between microcontroller and cpu/soc (ex. Stm32MP devices).

Consider using a stm32 with external memory controller, ex. 407Z E/G or 429Z E/G/I.
You can map parallel sram and parallel/qspi flash memories using the FMC/FSMC and access them like normal memory for almost everything, even execute code (although way slower).
In the 429 (and upper) you can also use SDRAM, that's the main difference between FSMC (flexible static memory controller) and FMC (works with dynamic memories too).

2 layer PCB is not a problem as long the memory is near the mcu. You have ~100MHz signals, not GHz.

Check the stm32f429 Discovery:

https://www.st.com/en/evaluation-tools/32f429idiscovery.html#documentation

The layout is 2 layer only, nothing impressive.
You have the gerbers, too.

[Siwastaja]

You may need to wait 1-2 years to be able to get STM32 microcontrollers again. For now, you need to limit yourself to only few parts that are available, most are not. Sucks.

In the long run, just bite the bullet and buy a part in QFP case and learn to solder it. It isn't that bad. Look at tutorial videos, etc. You will make it just fine.

Otherwise than that, STM32 devices are quite simple to design in. Leave a jumper for BOOT0 pin so it can be pulled up to Vcc to run factory bootloader, put a crystal and associated capacitors, get any 3V3 regulator, remember 100nF capacitors near all Vdd pins, bring out SWD programming pins but also bring out the UART pins that have the factory bootloader so you can reflash without special tools if needed through the same UART you would be using for development/debug/statistics/whatever.

Look at the actual pin mappings in the datasheet before deciding if you can use a part. If you need say 2 SPIs, 2 UARTS and 2 I2C's, chances are you can't get all of them at the same time even if the device sports 3 SPIs, 3 UARTs and 3 I2C's! If you plan to use DMA, you need to extend this process into looking at the DMA channel mappings in the reference manual, the available mappings may clash and prevent DMA to be simultaneously used for UART1 and SPI2, for example.

[SparkyTD]

How many of those actually need hardware SPI?

The display I guess, and maybe the ADC, depending on how often you do a sample. But I'd bet the bandwidth to the SD card would be low enough to bit-bang the SPI using GPIOs, and maybe the ADC also.

I think all three would need their own SPI interface. The display should remain responsive at all times, and the ADC should be able to sample the voltage waveform + seven channels of current waveform at at least 100-200 samples/cycle (~5-10ksps). The ESP32 is a great little chip, but I had to make some pretty serious compromises due to the lack of enough SPI interfaces.

[DavidAlfa]

You don't need spi interface for every device, that's Chip Enable/Select pin exist.

[Siwastaja]

SD card and display often get their own SPI buses for the obvious bandwidth reasons. Could be QSPI for the SD.

5-10 ksamples/s isn't much so that could likely be multiplexed with something else on the same bus.

But three SPI buses is not that many IMHO. Many mid-range devices have those available. I routinely use three SPIs because, well, why not. Multiplexing many on the same bus is like last resort when I run out of SPI busses.

[SparkyTD]

You don't need spi interface for every device, that's Chip Enable/Select pin exist.

Well, in my prototype the SD reader and the TFT display are on the same board, and they do share a single SPI interface, but the SD protocol is incredibly fragile, and I've run into many issues regarding chip-select, this is why I had to give up on the whole SD card thing for now. My code is mostly multi-threaded (e.g. display update and database service are on separate threads), and I got as far as using Mutexes to ensure that display updates and DB updates can't happen at the same time, but still, I have found that the SD was very very unreliable, and 9 times out of 10 it just wouldn't work. I'm not sure if this is because of bad Arduino libraries, or bad programming from my part.

Plus when I tried using SD with the ESP32, I used SPI, which isn't exactly the fastest thing. An STM32 offers more possibilities, like faster SD protocols (SDMMC/SDIO), and hopefully more reliability.

[DavidAlfa]

SD isn't that fragile, there're tons of projects doing that.
It's the spi sharing what needs to be done carefully, you're probably disabling Enable pin before the spi has finished with the last byte... Very common issue with the spi flags.

Single spi SD can achieve almost 5MB/s, not that slow.
4 bit SD requires a lot more protocol overhead, CRC, I wouldn't do that unless the stm32 has native SD interface.

[SiliconWizard]

As already said. Chip shortage is a major issue at the moment and picking an MCU that's available now and guaranteed to be so in a few months from now is almost impossible. (Well of course you can always overstock anything that fits your requirements and happens to be available right now, which a lot of companies do, which in turn worsens the situation...)

There's a number of STM32 parts that fit your requirements (no more than 2 MB internal flash though, as far as I could find) in the L4, F4, F7 and H7 series. In order of power consumption, roughly... pick what fits.

Right now, for instance, Mouser has the STM32L4R9ZIJ6 with 262 in stock. 2 MB flash, 640 KB RAM. BGA package.
In QFP package, you have the STM32F469BIT7: 2 MB flash, 384 KB  RAM. No stock though.
The STM32L4S9ZIT6 is in QFP package but in very limited stock...

Keep in mind that, apart from the availability issue, with those specs, the MCUs are not going to be cheap. The above parts are all around $20 or above.

OTOH -even though I'm not sure I would consider this at the moment for a commercial product - there's the RP2040, with 264 KB RAM, dual core M0+ and external QSPI flash. I'm pretty sure it would do the trick just fine for a smart meter. But availability in large quantities is pretty unknown.

In any case, you probably should look at MCUs using external QSPI flash. Because MCUs with >= 2 MB internal flash will limit your options drastically at the moment and will be kinda expensive too.

[SparkyTD]

In any case, you probably should look at MCUs using external QSPI flash. Because MCUs with >= 2 MB internal flash will limit your options drastically at the moment and will be kinda expensive too.

I would be okay with external flash, as long as I can find some that's not BGA (so I at least have a slight chance at soldering it myself), and if I can get away with a 2-layer design and easily find resources online for matching the trace-lengths.

I have given up on finding a part that JLCPCB has in stock, so everything BGA is out of the question.

I think I will just get an SMT32 dev board first to familiarize myself with the chip and with the programming before I design my own.

[SiliconWizard]

In any case, you probably should look at MCUs using external QSPI flash. Because MCUs with >= 2 MB internal flash will limit your options drastically at the moment and will be kinda expensive too.

I would be okay with external flash, as long as I can find some that's not BGA (so I at least have a slight chance at soldering it myself), and if I can get away with a 2-layer design and easily find resources online for matching the trace-lengths.

This isn't a problem. You can find a lot of QSPI flash chips in SO8 package. There's nothing really critical about routing QSPI signals either in general. Just keep the connections short. Exact length-matching is rarely required. We're talking about a quad SPI bus, most often SDR, at frequencies in the order of 100/133 Mhz most often.

[Siwastaja]

What are you doing with all the flash?

Storing some logging data in the internal flash? Don't you have the SD card for that?

Nice and large graphics for UI? Consider compression (or accept slightly lower image quality).

Complete large adventure games with quite nice-looking graphics, sounds and music fit in 2MB just fine in early 1990's.

Are you really sure this is a project that should run on a microcontroller? Maybe an application processor would be a better fit? Or an existing SBC?

In other words, what are you doing? How much can you tell about your project?

[SparkyTD]

What are you doing with all the flash?

Storing some logging data in the internal flash? Don't you have the SD card for that?

Nice and large graphics for UI? Consider compression (or accept slightly lower image quality).

Complete large adventure games with quite nice-looking graphics, sounds and music fit in 2MB just fine in early 1990's.

Are you really sure this is a project that should run on a microcontroller? Maybe an application processor would be a better fit? Or an existing SBC?

In other words, what are you doing? How much can you tell about your project?

The project is a smart electricity meter that should log consumption data from 7 channels with 1Hz frequency on an SD card, but one of the big challenges is the ability to also host a website that can serve that data quickly and reliably. For this I would need to use an sqlite3 database on the SD so I can request partial data, or to write more advanced queries. The webserver part was causing me a lot of problems (due to large libraries like jQuery and bootstrap) on the esp32 prototype, so that's why I would like to redesign the whole thing on a more powerful chip.

I've thought of building it around a Raspberry Pi Zero, but the chip shortage hit those pretty hard too, so they're even more difficult to find than STM chips.

As for the Flash requirements, my current ESP32 codebase that implements 80% of the features I want, compiles to a 1.1MB binary, so I need at least 2MB for extra headroom.

My current prototype is using a small 1.8" TFT display with no fancy graphics, and that would be enough for me, but the display still needs an SPI interface to be able to work in parallel with all the other devices.

[Siwastaja]

If you want to run existing networking, web serving and database stack, I'd do it (and have done similar projects) as a combination of any cheapest MCU doing the HW measurement things and communicating through UART with any SBC which runs standard linux for non-timing-critical web serving, logging, etc. Being standard linux computer with standard interfaces, chip shortage isn't a big problem, pick any SBC and write standard POSIX compliant code for it so that little or no porting is required (just to make UART work, for example).

Raspi isn't the only one. There are more robust and more industrial computers available, always been. Obviously, they cost more, but for one-off (or making a few dozen), does it matter?

For low cost, you would be aiming to produce hundreds of thousands of devices and should be able to afford a professional to make it all happen on the single cheap microcontroller. But then you really need to know what you are doing to be able to make the databases and web servers happen within, say, half a meg of flash. And do that reliably.

[SiliconWizard]

The project is a smart electricity meter that should log consumption data from 7 channels with 1Hz frequency on an SD card, but one of the big challenges is the ability to also host a website that can serve that data quickly and reliably.

Can you detail this a bit?

You mean that you'll want to access the meter via a web page when connecting to it on the network with a web browser, so that the device must act as a basic http server?
If so, this is not uncommon at all and you don't need all that much resources either. Unless you want it to be able to serve many client connections concurrently...
There are small http servers that are a good fit for that. Of course you should then probably avoid using things like PHP. Which, if the device builds its own web pages, wouldn't make sense whatsoever anyway IMHO. So maybe a little more details?

[Siwastaja]

Generally, you can simplify the HW requirements of small IoT gadgets by not making them full-blown web servers (that's problematic anyway - you'd need to install them in networks without firewalls to allow access to the web server from public Internet), but instead push the data using some simple interface into "cloud" i.e., the server somewhere else. That server elsewhere can then run whatever fancy database system to enable queries; and serve web pages for the whole planet.

[poorchava]

The first issue you have to solve is: where will you buy an actual STM32 in question?

[SparkyTD]

The project is a smart electricity meter that should log consumption data from 7 channels with 1Hz frequency on an SD card, but one of the big challenges is the ability to also host a website that can serve that data quickly and reliably.

Can you detail this a bit?

You mean that you'll want to access the meter via a web page when connecting to it on the network with a web browser, so that the device must act as a basic http server?
If so, this is not uncommon at all and you don't need all that much resources either. Unless you want it to be able to serve many client connections concurrently...
There are small http servers that are a good fit for that. Of course you should then probably avoid using things like PHP. Which, if the device builds its own web pages, wouldn't make sense whatsoever anyway IMHO. So maybe a little more details?

Yes, I want to implement a very basic http server that serves static files (i.e. no server-side processing like PHP), and the rest would be done in the client's web browser with javascript and a WebSocket connection to the MCU. I've tried doing this on the esp32 prototype, and it did sort of work with a very basic html file, but as soon as I tried adding large libraries like jQuery, amCharts and bootstrap, the esp32 started to struggle serving the files due to memory issues, and probably poor file handling (e.g. the minified jquery.js file is ~90kB).

It's okay if the web interface takes a second to load, it doesn't have to be instant, and also I'm not expecting more than 2 clients accessing it simultaneously. I think a more powerful STM32, let's say an H7 series should have no problem serving that web interface, given that I can find a nice and fast wifi chip.

[SparkyTD]

Generally, you can simplify the HW requirements of small IoT gadgets by not making them full-blown web servers (that's problematic anyway - you'd need to install them in networks without firewalls to allow access to the web server from public Internet), but instead push the data using some simple interface into "cloud" i.e., the server somewhere else. That server elsewhere can then run whatever fancy database system to enable queries; and serve web pages for the whole planet.

This is what I'm doing in my current working esp32 prototype. The website and the database are hosted on my AWS instance. The problem is that I really don't like the idea of internet-dependent gadgets, even if I'm in total control of the cloud service. Also, if I wanted to "debug" my home's excessive power usage by unplugging things and seeing how the consumption graph changes in real time (which was the original goal of this project), it would be very difficult because there are two different routers separating the meter from the internet, and if any of them was to go down, the whole device would become useless.

It would be much nicer if all of it was self contained withing the MCU, including the web server and the database, and it could act as a WiFi hotspot so I can walk around in the house with a tablet in my hand, testing the consumption of each individual wall plug and extension cord, while still being able to see the live data.

[SparkyTD]

The first issue you have to solve is: where will you buy an actual STM32 in question?

There is currently a H7 series chip with 5 units available on stock on one of the online electronics stores, with 220 more arriving in a few days. I am thinking about picking up a couple before it's gone for years, but at the same time, I'm also trying to figure out if it could even solve my problem, hence this forum post.

[apurvdate]

You can offload the display interface to a smaller AVR controller. Communicate AVR with your host controller via UART & update display via AVR SPI. Any arduino compatible AVR will ease your coding part. Additional IOs / peripherals can also support your host controller.

[Siwastaja]

You can offload the display interface to a smaller AVR controller. Communicate AVR with your host controller via UART & update display via AVR SPI. Any arduino compatible AVR will ease your coding part. Additional IOs / peripherals can also support your host controller.

No, don't do that. Using an undersized 8-bit microcontroller not really suitable for graphics, to generate graphics, then force yourself into the realm of multi-mcu design (keeping two firmwares in sync) makes absolutely zero sense when you can just pick any modern ARM microcontroller which comes with proper display abilities (including DMA) out of box and can do the graphics fine, and everything else at the same time.

[SparkyTD]

You can offload the display interface to a smaller AVR controller. Communicate AVR with your host controller via UART & update display via AVR SPI. Any arduino compatible AVR will ease your coding part. Additional IOs / peripherals can also support your host controller.

I don't think I have to do that. I plan on using a display with an SPI interface, so as long as the STM32 has a free SPI port, I see no need for offloading it to a second MCU.

[SiliconWizard]

Seems that the only real performance/memory factor that causes you questions here is this web server thing? Am I right?
As I suggested, while I think it's a good idea, you may want to work on that part a little bit to maybe make it less resource-hungry, so you have more options for an MCU.

As to the display, I don't see a single reason to implement it on another MCU either, since it's probably going to be relatively modest in terms of CPU use, at least I suppose so (and then certainly any very low-processing power one would not cut it anyway!), there shouldn't be any concern. As I see it, the web access will allow to give a pretty UI interface, while the display will be to just display the immediately useful information without needing to look particularly fancy.

[thm_w]

Yes, I want to implement a very basic http server that serves static files (i.e. no server-side processing like PHP), and the rest would be done in the client's web browser with javascript and a WebSocket connection to the MCU. I've tried doing this on the esp32 prototype, and it did sort of work with a very basic html file, but as soon as I tried adding large libraries like jQuery, amCharts and bootstrap, the esp32 started to struggle serving the files due to memory issues, and probably poor file handling (e.g. the minified jquery.js file is ~90kB).

It's okay if the web interface takes a second to load, it doesn't have to be instant, and also I'm not expecting more than 2 clients accessing it simultaneously. I think a more powerful STM32, let's say an H7 series should have no problem serving that web interface, given that I can find a nice and fast wifi chip.

A lot of sites just hotlink jquery.js, thats what I did for ESP8266.
Are you saying: you don't want to be dependent on an internet service, or you don't want to be dependent on internet being available at all?

[SparkyTD]

A lot of sites just hotlink jquery.js, thats what I did for ESP8266.
Are you saying: you don't want to be dependent on an internet service, or you don't want to be dependent on internet being available at all?

I don't want to be dependent on an internet connection at all. If I could find a memory-efficient way to serve ~1.6MB of javascript and some html from the SD card with reasonable speed, that would be perfect. Currently the biggest library would be echarts with a 1MB minified js file. Sure, I could probably find a smaller chart library, but I want my web app to look nice and modern. And the current cloud-hosted web app is already using this library.

[SparkyTD]

Seems that the only real performance/memory factor that causes you questions here is this web server thing? Am I right?
As I suggested, while I think it's a good idea, you may want to work on that part a little bit to maybe make it less resource-hungry, so you have more options for an MCU.

Yes, that is currently my biggest concern. The database solution is also a bit concerning, because in the current ESP32-based prototype the user can select all sorts of different time ranges on the consumption chart (hours, days, weeks, even an entire month) which means that in order to calculate the accurate kWh, it has to be able to quickly calculate the sum of up to millions of DB records. Even my PC with a relatively decent i7 CPU takes about ~300 milliseconds to query all the data necessary from an SQLite database file, and I'm worried about how a 480MHz ARM core would perform. I might have to look into implementing a custom, more efficient storage format.

[SiliconWizard]

I don't mean to judge and I don't really know how your system works at this point, but I'm just wondering if this amount of data (millions of records) is really needed for a smart meter.
You can probably get as useful and as accurate results while lowering your sampling rate a lot. Not sure you really need to *store* all this data.

[fchk]

I would split the whole system in two parts: a sensor part that grabs measurements and sends them periodically, and a processing unit that stores, processes and displays the measurements from one or more sensors.

The processing unit may be a PI or a small PC (NUC etc). For the sensor platform you may keep your ESP but keep the software there as simple as possible and just do measurements.

You also could use these sensors

https://www.eastroneurope.com/products/view/sdm120modbus (ok, these are 230V parts, but things like these should also exist for 120V systems)

or others, and you could later on process also other sensors like temperatures or water consumption etc etc.

fchk

[peter-h]

It is perfectly possible to implement a web server on a 32F4 device. I have one running here, and even TLS/HTTPS is almost working... The libraries out there for this stuff are full of bugs, and while lots of devs have independently fixed the bugs, they aren't publishing the fixes because they were paid by their employer to do this They probably spent a man-year just fixing ST's ethernet libs, polar SSL or EmbedTLS.

However, nearly all applications where data is being collected for retrieval elsewhere, upload data to a central server, from where it can be retrieved. This has various advantages e.g.

- simpler to run a web client than a web server
- can use a 3G/4G (or even GPRS) connection to the web
- IP doesn't have to be fixed
- no public visibility of the server port (80/443 etc) avoids constant hacking attempts by chinese/russian hackers
- no storage limit at the server
- no need to always be powered
- much more secure, due to no fixed IP, and any public network effectively placing it behind NAT
- can use nonstandard protocol to push the data to the server e.g. just bare UDP, avoiding all the TCP/IP complexity (no DHCP, etc)
- etc

You can set up your own server, which is very cheap these days, although you do need to have a backup strategy, and somebody needs to be in charge of that. But this is a well worn route - check out pricing at say Linode, for the virtual server options which most people use these days unless they need massive storage (terabytes). Or you can use e.g. Loggly https://www.loggly.com/ for a ready made server.

The hacking issue is very real. The IP will be discovered within hours, more quickly if you publish the DNS, and the server will be hammered with thousands of attacks per day. Eventually they will find a back door in your server code, which after all was just some old code with patches applied as they were found on forums

A smart meter may need to take fast measurements because it is calculating real time power (watts, and reactive VA) by V x I multiplication and that needs a lot of samples per 50Hz cycle. But you don't need to upload the data. Only aggregated samples.

[asmi]

SD card and display often get their own SPI buses for the obvious bandwidth reasons. Could be QSPI for the SD.

SD card does not support QSPI, only a single-bit SPI. SDIO != QSPI, it's an entirely different protocol.