Where or how do you get a secure embedded TCP/IP stack

[splin]

The vulnerability of TCP/IP stacks to attacks, including buffer overflow, malformed packets and DOS attacks is a significant concern - Amazon's freeRTOS being a prime (sorry!) example. This is becoming an important issue as goverments load on the pressure onto organisations, including power and water utilities, where significant disruption to society could result from their infrastructure being hacked. So what do you do if you want a secure, and preferably cheap, TCP/IP stack for an embedded product? It seems to me you have a few choices including:

a) Use a free product such as LwIP. This should be relatively quick to implement and have a small footprint having been designed for embedded products. But are they robust and maintained and tested against new attacks?

b) Port the latest Linux or BSD stack. I expect this to be a major undertaking, especially if you don’t need to, or have the resources to, support IPv6 so extracting the IPv4 code might be a nightmare. This has the advantage that Linux and BSD TCP/IP stacks are so widely used that they get a lot of attention from both attackers and defenders and any vulnerabilities get quickly patched.

c) Buy a third party stack. But how can you be confident that they are any more secure, and remain so, than free versions? The vendors will no doubt have a long list of reasons why (only) they can be trusted because obviously they have a team of the world's foremost TCP/IP and cyber security experts who are continuously engaged in probing and testing their S/W with the most advanced techniques to reveal any possible weakness, whilst diligently monitoring CERT alerts (and extensive inside contacts in various national intelligence agencies including GCHQ, NSA etc.)

But in the real world? How do you know that the product isn’t something that was written by ‘old Fred the comms whizz’ 25 years ago and has now retired and nobody at the vendor dares to touch beyond updating the copyright dates in the header files occasionally? If they do have a large dedicated team, chances are they are engaged in porting to innumerable different MCU/platform targets with their idiosyncratic tools and peripherals with little if no time to consider security. Especially as 'security' is very nebulous and virtually impossible to prove (prior to being hacked), whereas being able to tick the box 'Paduak MCS150C as a supported target' is likely much higher up a product manager's list of priorities.

Whatever the source of the stack, there is the vexed problem of keeping the firmware in an embedded product up to date. Clearly, having a TCP/IP stack means a large part of the update problem (distribution) should be solved (or at least simplified) but the remaining issues of managing the updates remain, including keeping track of the revision state of all products in the field and interoperability with earlier versions. Using a simple stack like LwIP presumably avoids this issue because it isn’t updated (but of course your porting might have errors which need to be fixed).

If you use a widely used stack then you probably need to be able to update quickly when vulnerabilities are found, which means a lot of cost in tracking updates by the developers, and porting those into your own if necessary, and distributing and managing your new releases. On the other hand if you use a (relatively) unknown or obscure stack then it’s possible that attackers will miss your unique vulnerabilities. I expect that this ‘security by obscurity’ does protect many embedded products - from casual attacks at least. But if your customer happens to be a high value target, such as a utility company sufficiently attractive for an attacker to focus their efforts on your stack, the ‘obscurity’ protection may become minimal or non-existent.

There are many more issues of course. So what do you do? Is the security weakness of TCP/IP stacks exaggerated? Falling victim to a DOS attack is likely nowhere near as severe as a breach allowing control/access to the device. Many of the high profile vulnerabilities of many consumer devices such as routers seem to be due to backdoors for developer or maintenance access, rather than in stack code itself.

What if your customer wants evidence that your product is secure - beyond bedazzling them with your gold-plated ISOxxxx/CMM level X quality systems and processes? How much does it cost to get an embedded product tested by a third party to 'prove' that it is secure?

And how could you rely on a third party's evaluation? Given the nature of attacks, testing requires creative and innovative approaches to testing as well as rigorous and methodical testing. The latter no doubt could be certified to ISOxxxx  (whatever standards developed by whatever committees) but that may add relatively little value to the former which requires individuals motivated to understand the mindset and techniques of attackers and constantly anticipate and investigate previously unknown or unused types of attack. These skills would be almost impossible to quantify in any sort of certificated form but are likely to be much more important than any size army of people with clipboards counting bytes, measuring stack depths, complexity metrics etc. etc.

[ejeffrey]

d) don't put microcontrollers on the public internet.  Microcontrollers either use a non-ethernet inteface (USB, serial, CAN, bluetooth) or go on private/isolated subnets. Access is controlled via proxies running on an a general purpose OS with a fully featured well vetted TCP/IP stack, firewall, TLS, and authentication libraries and protocol implementations.

Because honestly, Amazon's FreeRTOS vulnerabilities aside, the reason most IoT stuff is such security garbage has nothing to do with TCP/IP stack vulnerabilities.

[Berni]

In most cases sticking to something proven in the best way to do it. I suppose baking your own TCP/IP has the advantage of being obscure so that an exploit for a known stack doesn't work on yours. But there is only so many bad things you can do to a TCP/IP stack so you are bound to repeat a security hole that some other popular stack had many years ago. Even good programmers are not perfect and will miss something somewhere.

But very rarely is the actual transport layer at fault for an exploit. Sure DOS attacks are a thing but there is nothing you can practically do when someone is opening millions of connections to you or barfing so much data at you that your Ethernet link gets clogged up. Usually the layers sitting higher are at fault, they do more complicated work and are often very configurable by the user. The best mitigation for that is to run as few services as possible and keep them simple. This is easier if your device only opens connections to other devices and never listens for connections, you can simply ignore any inbound data that you didn't ask for. But in any case whatever application is using the TCP/IP is even more at risk of having a security hole.

Not running a OS quickly makes things easier to secure since a lot of services that are common attack vectors are simply not there. And since the exploit is likely going to be inside the application code you wrote on top of the TCP/IP stack means that the attack only works on that one product family.

More of an issue with linux systems is that they never get updated. A WiFi router might come out on release day with a few year old version of linux because that's simply the latest kernel they had for that low cost SOC chip. Because its an old chip the development of the linux kernel from the manufacturer disappears and nobody wants to port it themselves so the router continue being sold for 5 more years with the same linux image. Then the router keeps working at the user for over 5 years before it gets replaced...etc So you end up with Linux builds >15 years old running on devices constantly connected to the internet in vast numbers. Even if you wanted to keep your router firmware up to date you can't, new firmware simply was never created.

But yeah part of the problem is that when management is breathing down the necks of the programing team about getting it ready on some ridiculous deadline (Because they really need to hit that marketing window and they already wound up the hype train for the product). At that point the programmers are just happy to get the thing mostly working. Lots of shortcuts are taken, random code is slapped together, nobody has time to test it properly, the codebase becomes a big tangled ball of spaghetti that nobody wants to even touch once it works. By the end of all that security is pretty much the last thing on anyones mind, besides spending 5 months on security means no shiny new features in the product and how are they going to explain that to the investors.

[richardman]

Yup, exactly the kind of things I am alluding to in the FreeRTOS thread (sorry). As a third party vendor, it is unreasonable for us to write a TCP/IP stack ourselves - it takes too much resource that we do not have. So lwIP is appealing. There is a healthy developer and user community around it so hopefully there aren't any/many exploits like the ones in FreeRTOS. However, I think that the FreeRTOS incident gives a clear directive to us commercial vendors (and in theory non-commercial suppliers) that our customers need to understand the need and allow for security updates on their firmware, and that we must provide fixes to our software in a timely manner as well.

Of course from the user side, any updates could take a long validation time and they may not like to have to update hundreds, or thousands or more of their gadgets in the field just because there are exploits that may or may not affect them.

It may be interesting to see the feasibility of using the Linux or FreeBSD etc. stack, but chances are they make too much assumption about (having virtual) memory that will make them useful for MCUs.

There are a small number of proprietary TCP/IP stacks for MCU. uC/OS had one but they are now part of SiliLabs. CMX still has one, as does Segger. OTOH, security via code opaqueness may not be any better than open source ones either. It would be interesting to find out whether the FreeRTOS exploits were discovered with or without the benefits of having the source code available.

[nctnico]

Like ejeffrey already wrote: don't connect microcontroller gadgets to internet directly.
It makes a lot of sense to offload the network stuff onto a secondary processor. In my most recent IoT-ish project I used a Wiznet chip. Even if the network chip gets hacked the device itself can still function. But ofcourse the network chip could become part of an evil botnet.

I have a lot of experience with uIP (the predecessor of lwip from the same author) and it took a lot of effort to get it in a useable state. Based on that experience I have become wary of using TCP/IP network stacks and I wouldn't want to use Lwip in particular.

[mrpackethead]

Topical thread for me at the moment.

I'm considering options for an IP stacking on Microblaze ( Xilinx ).  Theres a lot of stuff using IPWP but not a lot with FreeRTOS.. However FreeRTOS+IP looks like it will perform cosniderably better.

[Geoff_S]

Based on that experience I have become wary of using TCP/IP network stacks and I wouldn't want to use Lwip in particular.

Curious about this - anything you can share about why the reluctance to use LwIP ?  Seems to be one of the more popular stacks for microprocessors.

[nctnico]

Based on that experience I have become wary of using TCP/IP network stacks and I wouldn't want to use Lwip in particular.

Curious about this - anything you can share about why the reluctance to use LwIP ?  Seems to be one of the more popular stacks for microprocessors.

uIP has been written very poorly to allow maximum optimisation to run on a 16 microcontroller. This made it extremely hard to follow an debug. If you look at the web page of the author (of lwip and uIP) you'll see most of his software are proof-of-concepts instead of production ready software. If you need to use a TCP/IP stack in today's world you need to use one which has been designed for security from the ground up.

[westfw]

the reason most IoT stuff is such security garbage has nothing to do with TCP/IP stack vulnerabilities.

Amen to that!
The main reason that some stacks have fewer flaws is only that their flaws have already been found.  :-(
(Hey @Legacy: any well-trusted TCP/IP stacks written in Ada or other "memory-safe" language?)

[Whales]

Aside on FreeRTOS: a few years back I read some dramas about FreeRTOS' license terms prohibiting benchmarking (and some other dramas).  Now I can't find this in their published licenses.  Have things changed?

EDIT: Old reference material https://www.freertos.org/FreeRTOS_Support_Forum_Archive/January_2015/freertos_FreeRTOS_Benchmarking_f78bbf5fj.html

[splin]

d) don't put microcontrollers on the public internet.  Microcontrollers either use a non-ethernet inteface (USB, serial, CAN, bluetooth) or go on private/isolated subnets. Access is controlled via proxies running on an a general purpose OS with a fully featured well vetted TCP/IP stack, firewall, TLS, and authentication libraries and protocol implementations.

But in many cases that isn't possible. For example, you have a remote water level monitoring station; whilst some water companies do operate private radio networks for these types of application, in many cases a GSM modem is the only practical communications option. It may have to run from a solar panel and batteries limiting average power to << 1W. You could meet that power budget with a 200MHz Cortex M4 or even a 400MHz M7 design with lots of memory and AES encryption built in so plenty of resource available for implementing a robust stack with proper encrytpion.

Running a full strength OS such as Linux or BSD would require something like a Raspberry Pi or similar which would be very difficult and probably impossible with << 1W. It would also significantly increase the BOM cost and the management costs as you now have to maintain the Linux/BSD software as well as your firmware. I expect reliability would be impacted adversely too. Most importantly there's no long term security of supply for modules like the Raspberry Pi.

And security wise you may have made matters much worse. Yes the stack is well tried and tested and security fixes are generally rolled out very quickly. But you have massively increased the attack space as you have to ensure the security of all the software running on your communications gateway. Obviously you would limit running services to the absolute minimum but still you now have to follow CERT advisories/alerts for far more than just the stack. I'd argue that a firewall is also unnecessary in an embedded product as it will either not accept any incoming connections from anyone, or have a single incoming port to which only your own (hopefully) secure F/W is connected. Blacklists and whitelists likely have no place here either because of the need to manage them remotely.

And yes a remote water level monitoring system is unlikely to be much of a threat to safety or national security should hackers gain control. Until the water company requests a remotely controlled output to operate the floodgates or whatever.

Because honestly, Amazon's FreeRTOS vulnerabilities aside, the reason most IoT stuff is such security garbage has nothing to do with TCP/IP stack vulnerabilities.

Yes I'm sure this is true, but stack vulnerabilities are still a concern - even if it just the issue of persuading your customer that your stack is secure. Perhaps the solution is to test against up to date attack tools like SATAN (ok. that's probably long obsolete). Are there any such test suites and procedures that are generally accepted to demonstrate a products resistance to attacks?

[NorthGuy]

I don't think it's possible to avoid DoD attacks. Attacker can just flood you with packets. To find out if a packet is good or bad you have to receive it. And you can only receive so match. Thus, a flood will cause downtime when a legitimate user has hard time connecting. May also discharge your battery during the night.

If you have "secure" things, such as SSL, this makes DoD attacks much easier - small amount of packets causes lots of work on the CPU site. This makes much easier for the attacker to blow up your power budget, or drain batteries during the night, possibly causing permanent battery damage or even fire. Overheating and killing the CPU may also be possible.

Your own stack is likely to be more secure, because you need a limited subset of what the regular stack provides, also because your foremost goal would be security. But it'll come at much greater price - you're lucky if you create a working stack in two weeks. Even if you create some vulnerabilities, they're unlikely to be exploited - it's a lot of work for a hacker. IMHO, 99.9% of hackers are not very far from Arduino users - they take a ready-made exploit and they apply it. Such attacks are of no concern to your custom stack.

[LapTop006]

Running a full strength OS such as Linux or BSD would require something like a Raspberry Pi or similar which would be very difficult and probably impossible with << 1W. It would also significantly increase the BOM cost and the management costs as you now have to maintain the Linux/BSD software as well as your firmware. I expect reliability would be impacted adversely too. Most importantly there's no long term security of supply for modules like the Raspberry Pi.

I've been playing with the chip in the pocketbeagle, with Linux running they idle at roughly half a watt. At load, well above unsurprisingly.

[legacy]

Hey @Legacy: any well-trusted TCP/IP stacks written in Ada or other "memory-safe" language?

 

[rstofer]

But very rarely is the actual transport layer at fault for an exploit.

Exactly!  Encryption, or the lack thereof, is not the responsibility of TCP or IP.  These two protocols just deliver packets and, in the case of TCP, guarantee delivery.  But it's just packets, the contents are determined at a higher level.  Buffer overflow isn't really possible with TCP because packet size is limited by the protocol.

The maximum TCP packet is 64k but the maximum transmission unit (MTU) of Ethernet is 1500 bytes.

I wouldn't be in a hurry to blame TCP or IP for security problems.  They're just the messenger, not the message.  Time is better spent getting SSL to work.

[NorthGuy]

With C ... you have nothing, you are alone, the language doesn't help, doesn't collaborate, it's all on your shoulders ...

It's not on your way.

[nctnico]

But very rarely is the actual transport layer at fault for an exploit.

Exactly!  Encryption, or the lack thereof, is not the responsibility of TCP or IP.  These two protocols just deliver packets and, in the case of TCP, guarantee delivery.  But it's just packets, the contents are determined at a higher level.  Buffer overflow isn't really possible with TCP because packet size is limited by the protocol.

The maximum TCP packet is 64k but the maximum transmission unit (MTU) of Ethernet is 1500 bytes.

I wouldn't be in a hurry to blame TCP or IP for security problems.  They're just the messenger, not the message.  Time is better spent getting SSL to work.

Nope. SSL is not some magic sauce you can pour over a piece of code and call it secure. It doesn't work that way. At a higher level you'll need to design the system securely right from the start. As a rule of thumb a well designed system is still secure without encrypting the messages.

[legacy]

It's not on your way.

of course, and ten bottles of whiskey are on my way thanks to Ada 

[SiliconWizard]

It's not on your way.

of course, and ten bottles of whiskey are on my way thanks to Ada 

 

[ejeffrey]

d) don't put microcontrollers on the public internet.  Microcontrollers either use a non-ethernet inteface (USB, serial, CAN, bluetooth) or go on private/isolated subnets. Access is controlled via proxies running on an a general purpose OS with a fully featured well vetted TCP/IP stack, firewall, TLS, and authentication libraries and protocol implementations.

But in many cases that isn't possible. For example, you have a remote water level monitoring station; whilst some water companies do operate private radio networks for these types of application, in many cases a GSM modem is the only practical communications option. It may have to run from a solar panel and batteries limiting average power to << 1W. You could meet that power budget with a 200MHz Cortex M4 or even a 400MHz M7 design with lots of memory and AES encryption built in so plenty of resource available for implementing a robust stack with proper encrytpion.

Sure, sometimes this is just not practical.  But a lot of IoT stuff is not particularly power constrained, and people use microcontrollers out of the belief that they are "simpler".  While the microcontroller is simpler, that just means you have to do the complex stuff yourself. 

I also don't think your example is very good.  Distributed sensor networks with GSM can use carrier provided VPNs to central gateway that is the only device allowed to access them.  This is a good idea no matter what architecture you use.  Also, you can easily design such a system and keep within a low average power.  You are already going to be operating with extensive use of sleep states since GSM modems take up a fair bit of power when operating.  A totally reasonable architecture is to have a microcontroller that is monitoring water levels and can wake up the "main" processor from a deep sleep mode either at regular intervals or when there is an alert.  The main processor would then start up the modem, read a batch of data from the micro and transmit it to the central server before going to sleep.  Many smart phones do this: they have a microcontroller that operates the accelerators and other sensors, and can wake up the main processor when needed.  If for some reason you couldn't use an external VPN, but had to implement the security on the sensor, this is the architecture I would suggest.

There is no one-size fits all approach, but I strongly believe the default answer should be "if it needs to run TCP/IP, use a full size CPU and OS", and you need to carefully justify any exception to that.

[legacy]

I wouldn't be in a hurry to blame TCP or IP for security problems.  They're just the messenger, not the message.  Time is better spent getting SSL to work.

Precisely 

I remember an implementation over the transport layer where the TCP/IP was merely used to transport messages, and it was OK, but there was a bug in a third library, and under certain conditions, a too long message was able to cause leakage of precious information in the application.

A funny episode 

[rstofer]

I wouldn't be in a hurry to blame TCP or IP for security problems.  They're just the messenger, not the message.  Time is better spent getting SSL to work.

Precisely. I remember an implementation over the transport layer where the TCP/IP was merely used to transport of messages, and it was OK, but there was a bug in a third library, and under certain conditions, a too long message was able to cause leakage of precious information in the application.

These leaks have always been a problem.  Perhaps more so with PCs but it was a really big deal in the mainframe world as well.

When multitasking allowed for multiple applications to be resident at the same time, protecting the data of one application from code in another application was a really big deal.

In some ways we see in crt.S some code to clear .bss on startup.  That's pretty straightforward when there is only one app and I assume, but haven't verified, that the same thing happens when multitasking.

Base and bounds registers were used on the CDC 6400 to keep apps in their own memory space.  It was all too easy to allocate a huge array and read the contents of some other dataset so the registers put a limit on the addressable space.  In addition, the allocated memory space was always set to zero when the app was loaded. 

Classified information and all that...

You would think this stuff would be settled by now but we just keep coming up with new ways to make old mistakes.

[richardman]

First of all, do not evaluate lwIP based on what one might know from uIP. Yes, it was originally written by the same author, but lwIP was started with a much better set of requirements and specification. Furthermore, it has been enhanced by many people since then.

Second, to use lwIP effectively, and probably any TCP/IP, using an RTOS is a must. I have no personal experience of FreeRTOS+lwIP. However, we do provide an RTOS with our lwIP package. Without an RTOS, you basically have to avoid the threading issues inherent with multiple execution context that exist in a TCP/IP environment. Any workaround to use lwIP bare metal would not be worth it to avoid using an RTOS.

[rstofer]

Second, to use lwIP effectively, and probably any TCP/IP, using an RTOS is a must. I have no personal experience of FreeRTOS+lwIP. However, we do provide an RTOS with our lwIP package. Without an RTOS, you basically have to avoid the threading issues inherent with multiple execution context that exist in a TCP/IP environment. Any workaround to use lwIP bare metal would not be worth it to avoid using an RTOS.

The original mbed platform (LPC1768) had lwIP as one of the libraries.  Using an RTOS wasn't mandatory and I don't think it was available at the time (as one of the standard mbed libraries).  I had very simple needs: Open a TCP connection to a LaserJet and send packets.  At some point the print job was complete so close the connction.  Nothing exotic, no replies expected, no worry about out-of-order reception (as there was no reception) and so on.

Under at least one very limited application, lwIP can be used on bare metal without an RTOS.

I just built a simple daughter card containing a MagJack and plugged in the mbed board:

https://os.mbed.com/platforms/mbed-LPC1768/

Beyond my very simple application, I think I would use FreeRTOS and the included TCP/IP stack (based on lwIP):

https://www.freertos.org/FreeRTOS-Plus/FreeRTOS_Plus_TCP/index.html

[nctnico]

You definitely don't need an RTOS to do TCP/IP and other networking stuff. To use an RTOS or not depends entirely on the scheduling requirements of the entire application. A network stack is just another interface which inputs/outputs data.

[richardman]

Perhaps my language was too strong, ha ha. Anyway, of course one can run lwIP without an RTOS. There's even a page on this
http://lwip.wikia.com/wiki/LwIP_with_or_without_an_operating_system

It's for 1.40 and earlier, but probably not too much different from 2.0. However, I still maintain that with an easy to use RTOS, there's no particular advantage of using lwIP that way in the general scenario.

[cdev]

There is no good reason to put critical infrastructure on the Internet and a great many reasons not to. Frankly, it should be illegal to put 'critical infrastructure' on the public Internet. I thought it was, actually.

So frankly, something is not adding up.

[rstofer]

But very rarely is the actual transport layer at fault for an exploit.

Exactly!  Encryption, or the lack thereof, is not the responsibility of TCP or IP.  These two protocols just deliver packets and, in the case of TCP, guarantee delivery.  But it's just packets, the contents are determined at a higher level.  Buffer overflow isn't really possible with TCP because packet size is limited by the protocol.

The maximum TCP packet is 64k but the maximum transmission unit (MTU) of Ethernet is 1500 bytes.

I wouldn't be in a hurry to blame TCP or IP for security problems.  They're just the messenger, not the message.  Time is better spent getting SSL to work.

Nope. SSL is not some magic sauce you can pour over a piece of code and call it secure. It doesn't work that way. At a higher level you'll need to design the system securely right from the start. As a rule of thumb a well designed system is still secure without encrypting the messages.

I don't see how a system can be secure without encryption if the packets can be intercepted.  Plain text is just that: plain.  Easy to read, etc.

Now, for an isolated network, not connected to the Internet, sure, it can be secure without encryption but that's the exception these days.  For better or worse, everything is connected to the Internet.  And everybody can read every packet, one way of another.  Just ask the NSA!

Which leads me to wonder what the utilities are thinking when they do connect to the Internet.  Encrypted or not!  It just won't end well.

[nctnico]

But very rarely is the actual transport layer at fault for an exploit.

Exactly!  Encryption, or the lack thereof, is not the responsibility of TCP or IP.  These two protocols just deliver packets and, in the case of TCP, guarantee delivery.  But it's just packets, the contents are determined at a higher level.  Buffer overflow isn't really possible with TCP because packet size is limited by the protocol.

The maximum TCP packet is 64k but the maximum transmission unit (MTU) of Ethernet is 1500 bytes.

I wouldn't be in a hurry to blame TCP or IP for security problems.  They're just the messenger, not the message.  Time is better spent getting SSL to work.

Nope. SSL is not some magic sauce you can pour over a piece of code and call it secure. It doesn't work that way. At a higher level you'll need to design the system securely right from the start. As a rule of thumb a well designed system is still secure without encrypting the messages.

I don't see how a system can be secure without encryption if the packets can be intercepted.  Plain text is just that: plain.  Easy to read, etc.

Then please read about how to implement a secure system and you'll see exactly why 1) encryption is only a very small part of security 2) secure systems need to be designed to be secure from the ground up. The very short explaination is that a secure system also 'senses' someone has broken in and is able to take counter measures.

[Berni]

Encryption does help protect you from people looking at your data and sending fake data to you, but only if everything is done correctly.

The more difficult problem is unexpected input data. The application layers can start getting pretty complex functionality and that vastly increases the chances that a weird input might cause it to do something stupid. This means weird unused ID codes in messages, massive data lengths in messages, negative numbers where there should be only positive ones, or even just selecting a very specific combinations of settings that causes the software to trip up somewhere down the line.

Even if you had a perfectly secure stack with heavy encryption that's correctly implemented you could still get hacked. So your encryption if flawless, but then someone hacks the device you are talking to and convinces it to send you a command with "Read username, length -1" It all goes fine trough the layers until the application layer on top of it goes check it and it does "if(length < sizeof(username))" so it looks just fine. It sends the username into the send function and it might be taking length as unsigned(since negative lengths don't make sense anyway) and so interprets -1 as being 4294967296 and simply starts sending out a 4GB long message containing basically all of the local RAM. Whops...

That was just one example of how something bad might happen. There are many many many other scenarios that can cause other bad things to happen. Often this application layer gets pretty huge and complicated and people don't tend to test it against silly nonsense inputs. Especially when the programmers are pulling an all-nighter to meet the deadline.

[legacy]

You definitely don't need an RTOS to do TCP/IP and other networking stuff. To use an RTOS or not depends entirely on the scheduling requirements of the entire application. A network stack is just another interface which inputs/outputs data.

edit:
You need an RTOS multitasking OS for a multi-socket application, e.g. a server-model waiting for several clients to be served 

[westfw]

Multitasking is very helpful.  "Real time" is completely unnecessary.(linux has a fine TCP stack and isn't real time.  Both cisco and Procket used run-to-completion schedulers.)

[forrestc]

You need an RTOS for a multi-socket application, e.g. a server-model waiting for several clients to be served 

Actually you don't.   

I am currently shipping RTOS-free microcontroller-based products which do not currently use a RTOS, and which support several HTTP clients being connected at the same time. 

There is a lot of misconception that a RTOS or OS is needed for this type of stuff.    Really what RTOS'es get you are things like being able to write code in run-to-completion style instead of cooperative multitasking style, better scheduling, and good inter-task communication (which simplifies message or event based programming).   There are other advantages as well, but being able to serve several network clients is not one of them.  In fact, the  highest-performance web servers on the web service all of their clients in a single task (per CPU).   This is actually a common performance-increasing strategy.  Search for "evented server" to see how common this is and to better understand why a servicing all of your clients in a single thread is more efficient. 

[nctnico]

You definitely don't need an RTOS to do TCP/IP and other networking stuff. To use an RTOS or not depends entirely on the scheduling requirements of the entire application. A network stack is just another interface which inputs/outputs data.

edit:
You need an RTOS multitasking OS for a multi-socket application, e.g. a server-model waiting for several clients to be served 

Nope. Not even under Unix. Look at how a typical BSD sockets select server works for example. All the connections are handled within the same thread. Often this is more efficient because you don't have the overhead of inter-process communication / data sharing (mutexes/semaphores) and task switching.

[legacy]

Nope. Not even under Unix. Look at how a typical BSD sockets select server works for example. All the connections are handled within the same thread. Often this is more efficient because you don't have the overhead of inter-process communication / data sharing (mutexes/semaphores) and task switching.

ehmmm with ucOS/2 on MPUs you don't have processes, and you don't have threads, and there is no fork(), and in the simplest working scenario (which I have used several time) tasks can even share variables on the global pool (of course with semaphores), or by passing messages, and it's less complex than all the stuff you have in Linux.

[legacy]

As a proof-of-concept the web server has to do is follow these steps:
- Listen on a TCP port
- Whenever a client tries to open a connection, (and there is a task able to serve it), accept it
- Parse the text sent by the client, HTTP request
- Process said request
- Reply a textual answer, HTTP response

I have to handle a maximal of six requests, therefore I have six equal  (static-) tasks waiting for a connection. They share the same code, but each task has its private context (local variables).

These tasks, plus an additional couple of tasks for the TCP/IP networking, are scheduled by ucOS/2.

What is wrong with this scheme? It's simple, it does its job, and the code is neat, and doesn't require to significantly refactor your code 

[NorthGuy]

As a proof-of-concept the web server has to do is follow these steps:
- Listen on a TCP port
- Whenever a client tries to open a connection, (and there is a task able to serve it), accept it
- Parse the text sent by the client, HTTP request
- Process said request
- Reply a textual answer, HTTP response

I have to handle a maximal of six requests, therefore I have six equal  (static-) tasks waiting for a connection. They share the same code, but each task has its private context (local variables).

These tasks, plus an additional couple of tasks for the TCP/IP networking, are scheduled by ucOS/2.

What is wrong with this scheme? It's simple, it does its job, and the code is neat.

It is nothing wrong with this. The OS here allows you to write your tasks in a linear fashion (as if other threads didn't exist). The OS does consume some resources and slows down things a bit in return, but people don't worry about these things nowadays. Without OS, you would have to write your tasks as state machines which get called when a relevant packet arrives.

You can do the same on the big HTTP server. You can use threads - each HTTP request runs in its own thread which is processed in a linear fashion. Or, you can use select() to poll. Polling is more efficient and will work faster, but your tasks will have to be written as state machines. Although select() limits the number of sockets you can poll. Therefore, it would be even more efficient if you got rid of select() and Berkeley sockets altogether and handled the packets by yourself. This way you could handle much more connections (50k is not out of limits), but you would need to write your own TCP stack.

[westfw]

with ucOS/2 on MPUs you don't have tasks...

...These tasks, plus an additional couple of tasks for the TCP/IP networking, are scheduled by ucOS/2.

Huh?  Does ucOS/2 have "tasks" or not?  Or were you using a more exacting definition of "task" the first time?

[richardman]

Huh?  Does ucOS/2 have "tasks" or not?  Or were you using a more exacting definition of "task" the first time?

uCOS definitely have tasks.

[legacy]

with ucOS/2 on MPUs you don't have tasks...

...These tasks, plus an additional couple of tasks for the TCP/IP networking, are scheduled by ucOS/2.

Huh?  Does ucOS/2 have "tasks" or not?  Or were you using a more exacting definition of "task" the first time?

lapsus, it doesn't have processes, it has static-tasks.

[legacy]

"static-tasks" means statically linked at compile time.

[Marco]

First rule of security ... never trust other people's code written in C. It's a well known fact that there are between 0-1 people capable of programming something securely in C, the 1 being you.

If it absolutely positively has to be written in C, HCC Embedded used MISRA compliant procedures to write one.

[westfw]

First rule of security ... never trust other people's code written in C. It's a well known fact that there are between 0-1 people capable of programming something securely in C, the 1 being you.

Meh.  1st rule is that you don't trust yourself to write security code.  "Don't trust anyone else" is the 2nd rule  :-(Rule 3 is that coders can't be trusted to understand cryptography, cryptographers can't be trusted understand code, few of either understands hardware, and none of them understand human nature.  :-( :-(

[SiliconWizard]

Oh yes. And don't forget:



 

[westfw]

That's "four"...

[richardman]

That's "four"...

 

[Karel]

I don't believe it exists.
Apart from giants like Google, IBM, Amazon, microsoft, etc,
a normal company does not have the resources to do this.
Besides, you very likely also need ARP, DHCP, firewall, etc.
It's simply too much work for a normal company.

If you really need network connectivity, use Linux and update often.

[hans]

I agree.

The state space of a modern TCP stack is so huge that you would spend ages to write verification tests for it. But let's not forget that vulnerabilities also originate often from the applications using TCP.

A little while back I heard a story from a lead engineer at a wireless SoC vendor, that they had tested their Bluetooth stack (that ran on a separate CPU in the SoC) to be robust against all forms of packet inputs. I'm not sure how they tested this - because even for bluetooth you can have a huge state space explosion that would have to verify against all inputs.
You get in the realms of formal verification, where you could write assertions or contracts (coming in C++20) that need to be proven never to be violated. If you then have some fallback behaviour (simplest would be send a RST packet and close the connection), I suppose you could have a reasonably robust TCP stack.

Of course you then still have the DoS attacks, as sanitizing inputs costs CPU time, which makes it easier to flood a device with requests or frames it needs to process.

Hence, don't put embedded devices publicly facing the internet or a big LAN.

[Marco]

As I said, there's HCC Embedded.

Or you can save the cash and use something written in Java, it rules out the source of 99% of exploitable bugs.

[legacy]

That's "four"...

it depends on the base and complement-type 

[legacy]

and none of them understand human nature.  :-( :-(

yes, understanding human nature is a very difficult and complex task.
See what has just happened here

Anonymous - Wed Oct 24, 2017
Yo men you have no chance. release the doc to those are skillful and retire from the scene, or look over there in the computer room, toothpick looks like it's gonna break in half any minute now.

On Xmas, someone found we had a vulnerability in our PHPBB forum and during a cyber attack it was used to damage the DB, among other things. Since then we are keeping everything segregated for our internal use, with weekly backups etc, and for sure we are not willing to spend more time at sanitizing our PHP interfaces.

We have updated Linux, Apache, and MySQL, and removed all the extra features from PhpBB. We should enforce it on a higher level and spend months at testing the engine for potentially new vulnerabilities, but it's for a hobby, therefore it's not worth with.

Now they are giving us ridiculous threats for releasing documentation we have under Confidential Disclosure Agreements, which of course they can only forget.

I doubted these individuals had ever really existed, now I know they actually exist and the Dorks are actually out of there 

[NorthGuy]

Apart from giants like Google, IBM, Amazon, microsoft, etc,
a normal company does not have the resources to do this.
Besides, you very likely also need ARP, DHCP, firewall, etc.
It's simply too much work for a normal company.

You're kidding? I've seen you posted your software on the forum. So, you can write software. I'm sure if you decide to write a TCP/IP stack, you will be able to do it relatively quickly. The task seems insurmountable to you since you don't know how to approach it, but it's really nothing difficult in TCP protocol and it is very well documented. Common attacks are also documented, so you'll be able to avoid them from the start.

[Marco]

The problem is doing it provably correct, or even doing it provably without buffer overflows when done in C.

[hans]

Getting something to a functional state is at most 10% of the work.

[legacy]

You're kidding? I've seen you posted your software on the forum. So, you can write software. I'm sure if you decide to write a TCP/IP stack, you will be able to do it relatively quickly. The task seems insurmountable to you since you don't know how to approach it, but it's really nothing difficult in TCP protocol and it is very well documented. Common attacks are also documented, so you'll be able to avoid them from the start.

It takes a lot of test cases, which sure it takes a lot of time and effort, but is feasible and it's an all matter of estimating how much time and effort, and this depends on the experience of guys in a testing team who know how to handle these scenarios more efficiently (because it's their job) than a common developer.

This is my professional experience: don't test your own code, ask a testing team to do it, it will be done better, and faster.

[NorthGuy]

The problem is doing it provably correct, or even doing it provably without buffer overflows when done in C.

You mean testing? Why is that a problem? Aside of simulations, you can connect it to a network, send all kinds of raw packets to it and make sure it handles everything correctly. Everything needs testing, how TCP is different?

[legacy]

You mean testing? Why is that a problem? Aside of simulations, you can connect it to a network, send all kinds of raw packets to it and make sure it handles everything correctly. Everything needs testing, how TCP is different?

eh, but it doesn't work this way 

A well-done testing activity done on engineering drafts requires you to stub the code, inject test cases and compare expected results with actual results, this must be done for every function, testing the behavior on both normal working and abnormal conditions.

If this passes, the code escalades to alpha and needs to be retested on the target (when applicable) to see how it reacts to stimulus in the real world.

Otherwise, you just have a shallow testing, which may hide surprises.

For the first step of the software life cycle, see at least what the following software are, and what you to do with them

• CodePurify
• Cantata

p.s.
and I am not talking about the dynamic coverage, which is mandatory in every beta steps for more advanced testing activities.

[NorthGuy]

You mean testing? Why is that a problem? Aside of simulations, you can connect it to a network, send all kinds of raw packets to it and make sure it handles everything correctly. Everything needs testing, how TCP is different?

eh, but it doesn't work this way 

Regardless of how it works, it's the same for TCP as for any other code.

[legacy]

Regardless of how it works, it's the same for TCP as for any other code.

sometimes you are embarrassing, like 200Mhz on a breadboard 

[Marco]

You mean testing?

No, if I did I would have said so.

When you code the stack in Java you know from principle it won't have buffer overflows. To do the same with C is not trivial, testing doesn't help much.

[NorthGuy]

When you code the stack in Java you know from principle it won't have buffer overflows. To do the same with C is not trivial ...

What you're saying sounds like utter non-sense to me. I guess someone may be afraid of buffer overflows and thus destine to program in Java, or worse in Python. I feel sorry for them. But why does it make writing TCP stack any different from writing any other code?

[NorthGuy]

sometimes you are embarrassing, like 200Mhz on a breadboard 

Oh, I actually had some free time, so I wrote some text which describes my 200 MHz breadboard tests for my blog, but I'm now busy with other things, so I had to pause. Writing blogs actually takes more time than I had imagined I'll post the link when I'm done.

[Karel]

The problem is doing it provably correct, or even doing it provably without buffer overflows when done in C.

You mean testing? Why is that a problem? Aside of simulations, you can connect it to a network, send all kinds of raw packets to it and make sure it handles everything correctly. Everything needs testing, how TCP is different?

Ofcourse you are right.
Google, IBM, Amazon, microsoft, etc. are so stupid and/or lazy that it took them many years to write a reasonable safe TCP/IP stack...

[Marco]

What you're saying sounds like utter non-sense to me. I guess someone may be afraid of buffer overflows and thus destine to program in Java, or worse in Python. I feel sorry for them. But why does it make writing TCP stack any different from writing any other code?

It's trivially exposed to attacker constructed inputs and when it's third party there is no security through obscurity.

When you have some proprietary code to take keypad input even if it has an exploitable buffer overflow it's not very likely to be abused (although it has happened with some console games).

[technix]

There is none - for the network stack to be secure it has to be fairly complicated due to all the checks and dynamic handling, and when you need those (as well as cryptography for authentication) the cost of that big a microcontroller start to increase so much that you might as well step up to something running full Linux and it would be a cost cut - a SoM like Raspberry Pi Compute Module or an integrated-DRAM SoC like Allwinner V3s or Microchip ATSAMA5D27C-D1G. With full Linux you get a well tested (since it is used in 99% of the world’s servers) and full functional TCP/IP stack.

[NorthGuy]

Google, IBM, Amazon, microsoft, etc. are so stupid and/or lazy that it took them many years to write a reasonable safe TCP/IP stack...

Companies are not people. They cannot be stupid or clever.

AFAIK, none of these companies ever created TCP/IP stacks (may be except Microsoft). I am not a historian, but I think Microsoft simply used old Unix TCP/IP stack without much modifications. They even used old names for the functions. Microsoft  WinSock API even has select() which is huge contrast to the rest of WinAPI..

The safe TCP/IP stack had already existed long before these companies were created (may be with exception of IBM).

[andersm]

When you code the stack in Java you know from principle it won't have buffer overflows.

Boundary checking is at best a partial solution. You still have an error condition that needs to be handled in a safe manner. Language will never be a solution to security. At best, it can reduce the risk of some types of bugs, at worst it opens up whole new classes of errors. Eg. with Java you add the whole JRE as a target, and it has had its share of vulnerabilities over the years.

There are some formal proofs of the TCP/IP protocol, but I couldn't find if a formally verified implementation exists.

[andersm]

I am not a historian, but I think Microsoft simply used old Unix TCP/IP stack without much modifications. They even used old names for the functions. Microsoft  WinSock API even has select() which is huge contrast to the rest of WinAPI..

I do not think this is correct. The Winsock API is obviously based on the Berkeley sockets API, to make porting easier, but I believe the implementation is their own. In older versions of Windows, some of the utilities (eg. ftp.exe) was based on BSD code, and contained the BSD license strings, but I don't remember ever seeing evidence that the same was true for the stack itself. It's always just been third- or fourth-hand claims.

[NorthGuy]

I do not think this is correct. The Winsock API is obviously based on the Berkeley sockets API, to make porting easier, but I believe the implementation is their own. In older versions of Windows, some of the utilities (eg. ftp.exe) was based on BSD code, and contained the BSD license strings, but I don't remember ever seeing evidence that the same was true for the stack itself. It's always just been third- or fourth-hand claims.

Of course there may be no direct evidence for a closed source product, so it is pointless to discuss, but look at these fragments from Microsoft's winsock.h:

Code: [Select]

* Basic system type definitions, taken from the BSD file sys/types.h.


Code: [Select]

* Structure used in select() call, taken from the BSD file sys/time.h.


Code: [Select]

* Commands for ioctlsocket(),  taken from the BSD file fcntl.h.


Code: [Select]

* Structures returned by network data base library, taken from the
 * BSD file netdb.h.  All addresses are supplied in host order, and
 * returned in network order (suitable for use in system calls).


Code: [Select]

* Constants and structures defined by the internet system,
 * Per RFC 790, September 1981, taken from the BSD file netinet/in.h.


Code: [Select]

* Definitions related to sockets: types, address families, options,
 * taken from the BSD file sys/socket.h.


[Marco]

At best, it can reduce the risk of some types of bugs, at worst it opens up whole new classes of errors. Eg. with Java you add the whole JRE as a target, and it has had its share of vulnerabilities over the years.

Sandbox escapes are only relevant if you allow arbitrary code execution within it. Those exploits are irrelevant to using Java as a programming language.

There are no classes of bugs which are as exploitable as buffer overflows and use after free. The only thing which gets close in economic damage is SQL injection and cross site scripting. Nothing Java has gets remotely close.

There are some formal proofs of the TCP/IP protocol, but I couldn't find if a formally verified implementation exists.

As I said twice already, HCC embedded.

[andersm]

Of course there may be no direct evidence for a closed source product, so it is pointless to discuss, but look at these fragments from Microsoft's winsock.h

All networking stacks that provide a Berkeley socket interface will have the same structures and types, since they're part of the API. Copying them from an existing source instead of typing them in yourself just makes sense.

[andersm]

There are no classes of bugs which are as exploitable as buffer overflows and use after free.

But they're not the only bugs. The idea that you're safe just because you're using Java died a well-deserved death in the 90s.

As I said twice already, HCC embedded.

In their brochures they're talking about using static analysis tools, but couldn't find any reference to any correctness proofs. If they actually have one I would have expected them to make a bit more noise about it.

[Marco]

The idea that you're safe just because you're using Java died a well-deserved death in the 90s.

Don't let perfect be the enemy of good ... like much of the IT industry has done for decades.

[legacy]

something running full Linux and it would be a cost cut

yup, or BSD, not necessary OpenBSD, but it would be an option.

[legacy]

The only thing which gets close in economic damage is SQL injection and cross site scripting

The risk is real. On our DownTheBunker website, we have recently experimented in person on a cyber attack which has seriously damaged the DB, and by investigating on the causes, we have then found open vulnerabilities exposed by the PHP code 

[legacy]

which is the best purchase ever? The full set of TCP Illustrated hardcover books

[Marco]

The risk is real. On our DownTheBunker website, we have recently experimented in person on a cyber attack which has seriously damaged the DB, and by investigating on the causes, we have then found open vulnerabilities exposed by the PHP code 

PHP has another common type of bug, type inference causing code to behave completely unlike intended.

Type inference shouldn't be in a web language ... that's for scripting throw away text parsers no adversary will ever be able to do anything harmful with, or programming a computer game. Programmers hate Java for much of the same reason it's secure. Its verbosity creates bugs, but it generally doesn't create exploitable bugs. Not all bugs are created equal.

[NorthGuy]

Type inference shouldn't be in a web language ... that's for scripting throw away text parsers no adversary will ever be able to do anything harmful with, or programming a computer game. Programmers hate Java for much of the same reason it's secure. Its verbosity creates bugs, but it generally doesn't create exploitable bugs. Not all bugs are created equal.

Languages do not create bugs. People create bugs.

[Marco]

People create more exploitable bugs in some languages than others.

[legacy]

and some people ... well

[..] your projects are rubbish and your sentences are on a retarded 5-year-old level, and you code like shit and the joke of a forum you have has more SQL errors than your mom has crabs, and it has so much Engrish, broken code, bad links, and poor optimization that [..]

some people are embarrassing, especially when they have all the time and the effort to look at vulnerabilities of your website and constantly try to put it down with several and different approaches, starting from their technical skills so you have cyber attacks, or with intimidations and tolling, so your team is so irritated that is prone to vote for a complete shut down

[..] retire deeply underground, into your fucking cave with your troglodytes [..] or look over there in the computer room, toothpick looks like it's gonna break in half any minute now.

hi guys,
looking for old XILINX CPLD XC95108 5V in PLCC84 package
I need Qty: 5-10 units. Let me know if you have for sale.
Thanks

Yeah I'll send you some... after I shit them out and mail them inside a pack full of horse turds.

What is really embarrassing is that it's the not machine, a bug, something you can handle some way, here the problem is some human beings, those who clearly need medical assistance from a good Psychiatrist to come back normals.

Can't we provide any medical assistance, on the DownTheBunker website we are only a little group of hobbyists, therefore are we supposed to spend an equivalent time to clean the shit that *those* individuals are prune to put in public trolling or spending time at enforcing anti-spam mechanisms and, even better, rewriting a forum from scratch?

jesus, how annoying the internet has become 

[legacy]

I mean, even with all the whistles and bells and protections enabled on a forum written in pure Java code running on a paranoid-security-level OpenBSD server, the resistance is futile with those individuals