Alternatives to LwIP?

[nctnico]

The legend says that TCP/IP stack is "so complex" that no one should even think about doing that from scratch. This is the advice you get everywhere, from everybody: "don't do it".

While probably somewhat true, the result of that attitude is we are stuck with a few implementations which everybody agrees suck more or less nevertheless. And despite the ratio of users : available implementations, there still is no decent documentation. Pretty weird.

I think this attitude needs to be challenged. Is the complexity of TCP/IP exaggerated after all? If it takes weeks and weeks of learning into how to use LwIP, why not try your wings with a full custom implementation? Obviously this is not for everyone. But for those smart guys; according to peter-h many smart people exist.

Code reuse is more cult than science. Best thing with code is, code is easy to discard and new code can be written. Working around poor and old designs is a lot of work.

In the 80s TCP/IP was hard. People didn't have enough background to make sense of what is needed, so the learning time could be huge. These days, if you've been working with networks in detail for some time, it isn't that hard. It not a job for the less able people out there, but for a capable person its an approachable job. It can get harder if you really want to crank the performance, but we are only talking about small scale systems exchanging modest amounts of data. Skills in compact design might be more interesting that skills in networking.

Compact design leads to hard to maintain & understand code which is exactly why the current 'popular' stacks are so hard to use. Where it comes to writing your own stack: it is not just tcp/ip but you'd need to support ARP & ping and UDP ethernet protocols. On top of that there has to be support for basic things like DHCP and DNS. But the most difficult part is to make the stack interoperable with all kinds of network devices. LWIP has been exposed to a broad range of devices and fine tuned over the many years it has been out there. So the likelyhood it doesn't play nice with a device is small. From my own experience with implementing protocols, the most time consuming part is dealing with devices that respond differently compared to what you expend.

[dare]

As I recall, it turned out to be Amazon AWS playing games with TCP retransmissions in a way that LwIP didn’t like

Interesting. How was it solved?

IIRC we made a change to the server code to avoid the behavior.  Since devices were already in the field, this was the best solution.

3 simultaneous network conversations (one each with a cloud service, a local coordinator device, and the end-user’s mobile device) using 12 network buffers totaling 7.5K in size

I think that's more buffering than a lot of people would have allocated.

Well, at least in this situation, their devices would have failed or performed poorly.  As I'm sure you know, running with very small amounts of network buffer space requires careful analysis of application behavior, and lots of testing.  We spent a lot of time optimizing this.

This is where a deep understanding of network protocols is essential.  And unfortunately this is not something that a better network stack can make up for (although a poorly implemented stack can obviously make this harder).

[SiliconWizard]

Regarding skills and "cleverness", being able to write things from scratch or reuse things made by others are IMHO equally challenging, neither is easy. Again, ease of reuse is a typical management misassumption while management fears novel solutions would be too difficult.

I couldn't agree more. Both are equally challenging - at least if you do things half "right". You'll face different issues.

Engineering management is tough. Managers tend to fall into the silver bullet trap, because it looks easier... on them.
As a manager, if you pick one well-known, industry-standard solution, even if it's suboptimal or even plain sucks, then you'll be safe personally if something goes wrong. No one in upper management will be able to blame your choices. Now if you decide to let the team develop their own solution and things go sour - then it'll be 100% on your shoulders, and both your team and you upper management will point fingers at you. Which option do you think managers tend to pick unless they are ready to find a new job?

Other than that, sometimes reusing some third-party code is not an option depending on context. If you're working in an environment in which you need to follow strict processes, a given code style/code guidelines and have to *validate* your software, then you often are not too keen on just reusing third-party stuff - it would make your life miserable - unless it comes with full guarantees and a lot of paperwork, which pretty much excludes anything but expensive "certified" commercial solutions.

[coppice]

Regarding skills and "cleverness", being able to write things from scratch or reuse things made by others are IMHO equally challenging, neither is easy. Again, ease of reuse is a typical management misassumption while management fears novel solutions would be too difficult.

Doing anything really well is not easy. Even if you are doing something that's not very challenging there will be other smart people doing a similar not very challenging thing, and you need to outperform them to win. Reuse should speed things up, but doesn't necessarily make high skill levels less of an advantage in producing a competitive result.

[dare]

I think we would all like a sockets like interface to the stack on every device.

Well, no so much for me.   After many decades of using the sockets API I’ve come to consider it a miserable and error-prone interface for anything but the most simplistic of applications.   As soon as you want to do something even slightly sophisticated—say, like making a UDP server that will respond to any client, including those using link-local addresses—then the arcana has to come out.  Even things that should be simple, like: Which sock_addr structure should I use? What should I do with the result from write()? or When should I call close() vs shutdown()? are subtle and confusing to newcomers.  Sockets perpetuates the illusion that networking is easy while quickly leading devs into territory that can get them in trouble.  I can’t count the number of times I’ve heard a dev say "I copied this example sockets code [because understanding how to implement it from scratch is just too hard] and now it isn’t working [because I don’t understand that my situation isn’t as simplistic as the sunny-day sockets example would have me believe]"

On top of this, there is no one definition of the sockets interface.  Linux, Android, MacOS, iOS—all have subtle differences.  And as far as I can see none of the embedded stacks have anywhere near full support for interface (e.g. the IP_PKTINFO / IPV6_RECVPKTINFO socket options).  LwIP’s socket interface sure doesn’t.

Circling back to the topic at hand, neither of the alternate APIs in LwIP are particularly good (netconn or "raw").  We choose to create a portable veneer over the raw API because it was the most efficient of the three and gave us greater control over how asynchronous I/O was handled which was important for our power-constrained devices.

[westfw]

we are stuck with a few implementations which everybody agrees suck more or less

Well, you should note that most of the complaints are of the form "The documentation is poor and it's hard to use", rather than "When I plugged in my device my whole LAN melted down, and then I got a call from the people who run the server at the other side of the country that I had crashed their system."  That sort of stuff used to actually happen.

And then there's 100+ pages of "requirements for Internet Hosts, Communications layers" that a vendor MIGHT be required to certify that their implementation meets...

[coppice]

I think we would all like a sockets like interface to the stack on every device.

Well, no so much for me.   After many decades of using the sockets API I’ve come to consider it a miserable and error-prone interface for anything but the most simplistic of applications.   As soon as you want to do something even slightly sophisticated—say, like making a UDP server that will respond to any client, including those using link-local addresses—then the arcana has to come out.  Even things that should be simple, like: Which sock_addr structure should I use? What should I do with the result from write()? or When should I call close() vs shutdown()? are subtle and confusing to newcomers.  Sockets perpetuates the illusion that networking is easy while quickly leading devs into territory that can get them in trouble.  I can’t count the number of times I’ve heard a dev say "I copied this example sockets code [because understanding how to implement it from scratch is just too hard] and now it isn’t working [because I don’t understand that my situation isn’t as simplistic as the sunny-day sockets example would have me believe]"

On top of this, there is no one definition of the sockets interface.  Linux, Android, MacOS, iOS—all have subtle differences.  And as far as I can see none of the embedded stacks have anywhere near full support for interface (e.g. the IP_PKTINFO / IPV6_RECVPKTINFO socket options).  LwIP’s socket interface sure doesn’t.

Circling back to the topic at hand, neither of the alternate APIs in LwIP are particularly good (netconn or "raw").  We choose to create a portable veneer over the raw API because it was the most efficient of the three and gave us greater control over how asynchronous I/O was handled which was important for our power-constrained devices.

I agree. The basics of sockets are simple, but the details are extremely confusing when you first encounter them. Error handling can be downright funky. The big benefits are a lot of people are familiar with sockets programming, and most code you might want to reuse will have been written for sockets.

[nctnico]

we are stuck with a few implementations which everybody agrees suck more or less

Well, you should note that most of the complaints are of the form "The documentation is poor and it's hard to use", rather than "When I plugged in my device my whole LAN melted down, and then I got a call from the people who run the server at the other side of the country that I had crashed their system."  That sort of stuff used to actually happen.

Yeah. I recall crashing a high-reliability, core infrastructure grade telephone switch by sending it a malformed packet over an ISDN link  Only a power cycle of the interface would make it recover.

[Jeroen3]

My recommendation after attempting a few projects LwIP was that if your project requires more then only a few UDP packets you should use a linux based single board computer or soc (eg: rpi cm4 or wiznet).
The current trend with encryption everywhere only makes this more relevant.

[peter-h]

That is true for "knocking up working hardware" quickly, and the chinese do that a lot, but it is most suitable for short-life products, after which you get constant production hassles. And unless the CPU is fast, with lots of RAM, you end up with a painfully slow box.

Whereas if you build an ARM32 box with LWIP, you should get 10 (and possibly 20, with some "last time buy" stock grabbing) years out of the design. I've got 25 years out of a H8/323 box (with considerable LTB grabbing).

TLS is something else, with crypto suite fashions changing faster than a *****'s knickers But if you make a suitable field upgrade provision, then you will have control for

- as many years as the crypto can be run in software (already a 32F417, not that old, has no "useful" hardware crypto apart from AES)
- as many years as somebody in your company is able to maintain a machine for rebuilding the firmware (that is what kills many products in reality, as people leave, etc)

I am no expert but it's already clear to me that you have to pick your market.

- retail IOT, talking to a 3rd party server -> you will get shafted unless you build a "PC"
- retail IOT, talking to your private server-> you can do what you like (even IPV4 will be around for ever)
- industrial IOT -> talking to your private server -> got to do crypto to impress the customer but basically whatever you want in terms of details
- industrial IOT -> talking to anything else -> got to build a "PC"

My box is #3, with dire warnings to not use it for the others