Author Topic: What made you choose an RTOS, Linux, or both RTOS and Linux for a real product  (Read 9144 times)

0 Members and 2 Guests are viewing this topic.

Online Siwastaja

  • Super Contributor
  • ***
  • Posts: 11043
  • Country: fi
If you want a lesson in how to "decouple" code corrected the litmus test is simple.  Take every single c file in your project and write a "Unit test" for it.
I don't think that will help.

It's a good thought experiment, something to keep in mind, but it cannot be an absolute rule. Complex capabilities grow from simple units working together; like, A * B * C > A + B + C. So by definition, the coverage you can get from unit tests will be smaller. Which is why larger integration tests are needed.

But indeed, if you find out that nearly every of your module depends so heavily on others you can't test anything on those alone, it suggests there's a problem. Not just a testing problem; it's also a reuse problem, and it means the project is more difficult for a new employee to understand. And then again, overly modularized, with perfectly isolated units, can be equally difficult. I remember seeing projects where I spend a few hours poking at the codebase and not getting any idea how any normal flow would materialize through these trivially too small units. If every unit is small and contained, that means the business logic is hidden or distributed - hard to figure out. Small modules are easy to unit test (and tick a box), but these unit tests don't reveal anything.
« Last Edit: August 01, 2026, 09:26:31 am by Siwastaja »
 

Offline peter-h

  • Super Contributor
  • ***
  • Posts: 5925
  • Country: gb
  • Doing electronics since the 1960s...
Quote
if you remember peter-h's posting history, all sorts of very hard to debug "my networking dies every 5 hours"  issues in projects he inherited and tried to manage have been common. It's the exact pattern of having too much non-understood complexity, and the struggle is very familiar to me: it's a HR issue, how to find capable people who know what they are doing and can manage the complexity, understand difficult things, and write good, robust code. For our company, also struggling to find the good professionals, AI was finally the solution.

Here we go again. If you can't post something specific and informative to educate others, a personal attack is always a good solution.

Like I've said before, this shop is just around the corner and I suggest you drop in and buy one :)



I've been using RTOSes since the 1980s. Wrote my own for Z80, Z180, Z280. Even posted the code here. Now using FreeRTOS on 32F4xx.

The "own the machine" is about code structure. Much easier to write good structure, and if you get that right, you end up with a much more robust product because there are far fewer places for bugs to hide.

Did I ever post my networking dies every 5hrs? There are lots of Peters, mostly aged 65+. Maybe you are getting me confused.

The products I sell are extremely solid; thank you.
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline NorthGuy

  • Super Contributor
  • ***
  • Posts: 3465
  • Country: ca
This drastically simplifies programming. You don't need funny loops, horrid state machines which nobody can understand, etc.

This is a question of habit. If many years ago you took a different turn, you could be swearing now that you don't need funny threads and mutexes which nobody can understand.

Human brain works that way. If you repeat something over and over it gets easier and easier. This " ... makes us rather bear those ills we have than fly to others that we know not of"

In reality, event driven programming is not any harder (or easier) than consecutive threads.
 

Offline peter-h

  • Super Contributor
  • ***
  • Posts: 5925
  • Country: gb
  • Doing electronics since the 1960s...
Everything is a question of habit :)

An expert of Cobol-66 will out-do a novice in C :)

Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Online SteveThackery

  • Super Contributor
  • ***
  • Posts: 3220
  • Country: gb
  • 50 year novice
Speaking as a novice and hobbyist, I've very much enjoyed using FreeRTOS. The idea of separating logical functions out into tasks, with communication between them by messages, seems to help me structure my code better. Modularity and the reduction of inter-dependencies feels like a cleaner way for a novice like me to write code.

Of course, anything implemented using an RTOS can also be implemented with super-loops and state machines. Or you can make the whole thing using interrupts and events. Or all the other methods I can't think of just now. You can get as close to the metal as you like. Any functionality can be implemented in any methodology.

In this thread you've seen people advocating for whatever their preferred programming method is, and that's fine - they all work, and at the end of the day it's just what works best for you.

I should repeat that I am just a hobbyist with little experience of RTOSs, so my opinions aren't worth much. But I have enjoyed my first forays into FreeRTOS very much. My earlier experience was with Arm mBed. The programming experience was pretty horrible, but thankfully mBed has been abandoned.
 

Offline peter-h

  • Super Contributor
  • ***
  • Posts: 5925
  • Country: gb
  • Doing electronics since the 1960s...
I could not agree more.

But there is something else: can you or somebody else pick up your code in a few year's time? The way you are doing it, the structure should be obvious.

I've done state machines. One can do awfully clever ones. Did a lot of that with 22V10s and FPGAs as well ;)

Interrupts are needed for some things e.g. regular DAC stuffing (arguably timer -> DMA is better for that; zero CPU load). And getting data out of UARTs which (mostly nowadays) have no decent FIFOs.

I've known tons of clever programmers. Way above my league. But the stuff they do is near impossible to maintain. Especially those who love pointers ;)

FreeRTOS is brilliant. No trouble with it at all. I now have a load of tasks hanging off mine. Some of them are pretty big.



The key to making it all run nicely is, whenever there is nothing to do, do osDelay(10) or whatever...

Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 
The following users thanked this post: 5U4GB

Online Siwastaja

  • Super Contributor
  • ***
  • Posts: 11043
  • Country: fi
But there is something else: can you or somebody else pick up your code in a few year's time?

This IS important, and one of the key reasons one would take a linux box / SBC - finding people who understand how to get it working somehow again, or make it run on a different SBC, is relatively easy, compared to, say,  FreeRTOS + embedded networking (say lwip + mbedtls) specialists.

AI has totally changed the game though. It has no problem picking up interrupt-based, superloop based, event based, parallel thread based, mbedtls, lwip, linux, Raspberry Pi or whatever project.
 

Offline peter-h

  • Super Contributor
  • ***
  • Posts: 5925
  • Country: gb
  • Doing electronics since the 1960s...
If you build something around a linux box, everybody and their dog will know right away how to hack into it.

Unless a real expert sets it up... then you are back to specialisation.

And the hardware needed to run it goes obsolete much faster than embedded.
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Online nctnico

  • Super Contributor
  • ***
  • Posts: 30061
  • Country: nl
    • NCT Developments
If you build something around a linux box, everybody and their dog will know right away how to hack into it.

Unless a real expert sets it up... then you are back to specialisation.

And the hardware needed to run it goes obsolete much faster than embedded.
Not really. You just need to use a SOC from the right manufacturer. TI and NXP have SOCs with up to 15 years guaranteed availability. The IMX27 series which was introduced in 2008 is still in production. The IMX51 series (introduced in 2010) is still marked active.
« Last Edit: August 01, 2026, 06:28:27 pm by nctnico »
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline peter-h

  • Super Contributor
  • ***
  • Posts: 5925
  • Country: gb
  • Doing electronics since the 1960s...
Every consumer linux box I've seen was incredibly slow.

When Draytek went dray-os to linux, their stuff became painfully slow in the UI. Same with all the Linksys stuff. Same with everything else I've seen; got a load of ETH-4G boxes which are like that. The "multitasking" in linux seems to be just really poor. Or maybe crap app writing?

Embedded stuff is mostly "instant".
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 
The following users thanked this post: Siwastaja, 5U4GB

Online PlainName

  • Super Contributor
  • ***
  • Posts: 8683
  • Country: 00
But there is something else: can you or somebody else pick up your code in a few year's time?

This IS important, and one of the key reasons one would take a linux box / SBC - finding people who understand how to get it working somehow again, or make it run on a different SBC, is relatively easy, compared to, say,  FreeRTOS + embedded networking (say lwip + mbedtls) specialists.

I think it's important to specify whether one is trying to fix hardware-related issues (so, perhaps drivers) or logic issues. An incoherent program can be hard to decipher whatever platform it runs on. At least an RTOS-based system has a quite limited API to grapple with compared to Linux (or Windows).
 

Online nctnico

  • Super Contributor
  • ***
  • Posts: 30061
  • Country: nl
    • NCT Developments
Every consumer linux box I've seen was incredibly slow.

When Draytek went dray-os to linux, their stuff became painfully slow in the UI. Same with all the Linksys stuff. Same with everything else I've seen; got a load of ETH-4G boxes which are like that. The "multitasking" in linux seems to be just really poor. Or maybe crap app writing?
Just crap app writing. Most likely they use really slow (cheap) flash for the filesystem which then holds back everything. That is not a Linux problem but a hardware problem. If you use FreeRTOS you'll have the exact same problem (been there, done that).
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Online SteveThackery

  • Super Contributor
  • ***
  • Posts: 3220
  • Country: gb
  • 50 year novice
But there is something else: can you or somebody else pick up your code in a few year's time? The way you are doing it, the structure should be obvious.

At risk of just repeating what @peter-h said, I would take the above sentence and put "you" in bold. For hobbyists like me the chance of someone else picking up my code is vanishingly small. But loads of times I go back to a project after a few months or years. Sometimes I'll look at the code and can't even remember writing it.

But it has taught me something important: make your code as easy to follow as you possibly can. Write it for your future self. Comments cost nothing; descriptive variable names cost nothing.

I would even go further: if you are writing in C or C++, don't use pointers at all (apart from setting up interrupt vectors and such). I'm referring to '*' and '&'. Both languages are Turing-complete without them, so there is always another way, and almost always that other way is easier to follow. This advice is controversial! - I'm sure I'll get roasted. 😀
 

Offline peter-h

  • Super Contributor
  • ***
  • Posts: 5925
  • Country: gb
  • Doing electronics since the 1960s...
In C, you need to use * and & to pass values by address (i.e. efficiently) but that is not the same thing as pointer manipulation which is responsible for many bugs and opaque code.

Quote
Just crap app writing. Most likely they use really slow (cheap) flash for the filesystem which then holds back everything.

OK, but then probably 90% of these products have code written by "kids" and have hardware designed by complete novices :) They probably just put in 10MB FLASH and set up a 1GB swapfile :)

Maybe it is just the current world i.e. if you want to hack something together fast, you make a linux box, put openwrt (or whatever it is you want) on it, and you are done. It works well enough, but with the UI running like Compuserve on a 2400 baud modem :)

But Linksys and Draytek have been at it for many years...
« Last Edit: August 01, 2026, 09:20:24 pm by peter-h »
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline NorthGuy

  • Super Contributor
  • ***
  • Posts: 3465
  • Country: ca
I would even go further: if you are writing in C or C++, don't use pointers at all (apart from setting up interrupt vectors and such). I'm referring to '*' and '&'.

Is that Ok to use '*' for multiplication?
 

Online nctnico

  • Super Contributor
  • ***
  • Posts: 30061
  • Country: nl
    • NCT Developments
I would even go further: if you are writing in C or C++, don't use pointers at all (apart from setting up interrupt vectors and such). I'm referring to '*' and '&'. Both languages are Turing-complete without them, so there is always another way, and almost always that other way is easier to follow. This advice is controversial! - I'm sure I'll get roasted. 😀
I partly agree with that. The problem is that avoiding to use pointers in C is impossible to avoid; pointers are a core part of C. In C++ however you can write code without ever needing pointers. Just pass variables by reference (and use const to make them read-only if necessary). And if your platform can use STL (with containers like vectors, list and maps) and Boost, then you can write applications like you are writing it in Python.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Online PlainName

  • Super Contributor
  • ***
  • Posts: 8683
  • Country: 00
Quote
like you are writing it in Python

Not sure that's a good recommendation!
 

Online nctnico

  • Super Contributor
  • ***
  • Posts: 30061
  • Country: nl
    • NCT Developments
In C, you need to use * and & to pass values by address (i.e. efficiently) but that is not the same thing as pointer manipulation which is responsible for many bugs and opaque code.

Quote
Just crap app writing. Most likely they use really slow (cheap) flash for the filesystem which then holds back everything.

OK, but then probably 90% of these products have code written by "kids" and have hardware designed by complete novices :) They probably just put in 10MB FLASH and set up a 1GB swapfile :)
Not really. Cheap flash has erase times in milli-seconds. With a proper Linux distribution you have a proper filesystem and caching. This is something cut down Linux distributions lack and hence file system performance is on par with a 1990's era hard drive. Another problem could be using Python to run the UI. Starting Python on a slow filesystem and slow processor takes ages. Not so long ago I wrote a Python tool for an embedded device running a Buildroot based Linux but it took ages to start so I rewrote it in C++.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline EVblog1Topic starter

  • Regular Contributor
  • *
  • Posts: 131
  • Country: 00
I'm still following this thread, and I still feel that it would be really helpful if someone could share a specific example from one of their projects and explain the reasoning behind the design choice. I think that would benefit not just me, but anyone else reading this thread.

People could then ask follow-up questions, and the person sharing the example could answer based on their own experience. Right now, most of the discussion is at a general level, and I personally find it difficult to draw any concrete conclusions from that.

That said, this is just my opinion. I'm not trying to tell anyone how they should answer or asking anyone to share anything they're not comfortable sharing. If you'd rather keep your reply general, that's completely fine. I'm simply saying that, for me, a real example with the reasoning behind the design choice would be much easier to understand.

That's the main reason I started this thread I wanted to understand the kinds of real project requirements
 

Offline uer166

  • Super Contributor
  • ***
  • Posts: 1263
  • Country: us
I'm still following this thread, and I still feel that it would be really helpful if someone could share a specific example from one of their projects and explain the reasoning behind the design choice. I think that would benefit not just me, but anyone else reading this thread.

People could then ask follow-up questions, and the person sharing the example could answer based on their own experience. Right now, most of the discussion is at a general level, and I personally find it difficult to draw any concrete conclusions from that.

That said, this is just my opinion. I'm not trying to tell anyone how they should answer or asking anyone to share anything they're not comfortable sharing. If you'd rather keep your reply general, that's completely fine. I'm simply saying that, for me, a real example with the reasoning behind the design choice would be much easier to understand.

That's the main reason I started this thread I wanted to understand the kinds of real project requirements

Sure, here's my example: a handheld custom texting/locator radio with a monochrome display, a LoRa radio with a fully custom radio stack, and a GNSS receiver.
The high-level purpose is to be able to track other people that also have these radios, text each-other, and do basic navigation. The original firmware was a main 100ms ISR loop with a extra USART Rx ISR for GNSS data reception.

The main loop was doing:
  • The full radio stack with outgoing message FIFOs and reading incoming messages off the LoRa IC directly
  • Synchronizing itself to GNSS PPS pulse by simply reading it as a GPIO
  • Drawing the display graphics into framebuffer and configuring DMA transfer over SPI to display
  • Reading the GPIO buttons for user UI
  • Maintaining the full UI state machine in the menus/submenus

The protocol itself was TDMA and needed to be synchronized to the 1 second PPS, which in this architecture can be done to a +-100ms jitter level since the ISR itself runs every 100ms. There are guard times around radio transmit and receive that tolerate up to 300ms of jitter, so that particular hard realtime constraint was respected.
The UART RXNE ISR was a separate asynchronous ISR that would parse the incoming GNSS data from the UART and populate atomic values such as latitude, longitude, time, etc. Those are then read directly by the 100ms loop ISR without mutexes or any other primitives.

Notice the limitations that need to enforced:
  • The main screen render loop *has* to finish in under 100ms (well 100ms - whatever other time the code needs).
  • The 100ms jitter for radio stack vs absolute GPS time means it eats a bit into the total guard time budget, but 300ms allocated to it is very large overall so not a problem
  • The large guard time reduces the total on-air time of the radio artificially
  • The buttons can only be sampled every 100ms, so if you press and release one super quickly, the input may be missed, which is a minor annoyance

Now with the limitations in mind, I wanted to upgrade a few things and enable less guard time (more on-air time), rendering of maps that may take more than 100ms, and fix some UI input annoyances. Instead of the 2-ISR architecture I made it 3-ISR/context:

  • Periodic 50ms ISR running the radio stack that is hardware-synchronized to the PPS 1Hz pulse via a timer repetition counter: reduces jitter from 100ms to ~single digit us
  • Periodic 100ms UI ISR: deadline no longer critical
  • Asynchronous USART RXNE ISR doing the GNSS data reception, unchanged

Note that the split here is pretty intuitive: the radio stack does the radio, the UI context does UI, HOWEVER the amount of extra complexity that I had to introduce here is a bit disproportionate:
The radio stack doesn't run when there is no GNSS PPS pulse, which in some ways that is nice, but it needs extra attention since the actual code execution is no longer a continuous loop.
The data is no longer inherently atomic: the radio messages, both incoming and outgoing are now transferred via lock-free FIFOs, while the GNSS data is still inherently atomic. No mutexes are required here, and neither are critical sections (i.e. disabling ISRs).

Note how if I used an RTOS to make a similar architecture, very little would actually change: you still need FIFOs to transfer the data, you still need some way to maintain GNSS data consistency from the USART context to the others. FreeRTOS would not help me synchronize my 50ms radio stack to the PPS input, that still has to be purely hardware for zero-added-jitter. Perhaps one value it would add is maybe automatic time/performance measurement of the main UI loop as well as the radio stack loop, but that is like 3 C code lines for each context here that I have to store a freerunning counter value for performance validation anyway.

FreeRTOS specifically does not have lock-free FIFOs from what I remember, it uses critical sections to maintain data consistency in its' _from_ISR API implementations, so technically that would add some extra hard to measure jitter to the UART ISR (that *still* has to be an ISR, it cannot be a task), however it's certainly not a huge issue in this particular example, just an extra annoyance. In any case with FreeRTOS, you still have to maintain context relationships, keep track of deadlines, and make sure the state machines don't have any weird edge cases. I didn't use FreeRTOS here because it's a shitload of extra code and API to learn and get good at for very little gain, it would provide no abstraction of anything on its' own (that's what ST HAL would do in my case anyway), and of course overall it would be *less* deterministic.

One thing that I wouldn't have to write with FreeRTOS is a queue/FIFO implementation, but the whole thing is like 30 lines of C in a header-only library, infinitely reusable, and again lock-free which has tangible benefits for the extra-hard jitter/deadlines.
« Last Edit: August 02, 2026, 01:50:33 am by uer166 »
 

Offline 5U4GB

  • Super Contributor
  • ***
  • Posts: 1684
  • Country: au
But it has taught me something important: make your code as easy to follow as you possibly can. Write it for your future self. Comments cost nothing; descriptive variable names cost nothing.

One thing I've found really good for this is Claude, which reads comments when it analyses code. If it says something like "this code appears to be doing X" when it's not doing X at all then it's a sign you need to improve the comments. There's a number of places where I thought my code was reasonably well commented but to a third party, or myself coming back a year or two later, it wasn't. It acts like a proofreader for code/comment comprehensibility.
« Last Edit: August 02, 2026, 05:33:49 am by 5U4GB »
 

Online Siwastaja

  • Super Contributor
  • ***
  • Posts: 11043
  • Country: fi
Every consumer linux box I've seen was incredibly slow.

And not only slow, usually unstable / buggy as well.

The linux SBC is like a poorly maintained vintage car - it breaks down all the time and consumes a lot of fuel, but then again "anyone can fix it". That's the appeal. If your (or mine) MCU-based FreeRTOS solution has a serious bug or needs a large design update, the number of people who can do it is one thousandth of those who can reinstall a newer Debian or write a bash script or make systemd launch something at boot.
 

Offline 5U4GB

  • Super Contributor
  • ***
  • Posts: 1684
  • Country: au
The linux SBC is like a poorly maintained vintage car - it breaks down all the time and consumes a lot of fuel, but then again "anyone can fix it". That's the appeal. If your (or mine) MCU-based FreeRTOS solution has a serious bug or needs a large design update, the number of people who can do it is one thousandth of those who can reinstall a newer Debian or write a bash script or make systemd launch something at boot.

Or to put it another way, "any idiot can slap together a device running Linux, and they frequently do".
 

Offline peter-h

  • Super Contributor
  • ***
  • Posts: 5925
  • Country: gb
  • Doing electronics since the 1960s...
Given the large firms that do a crap job, getting a responsive linux box working is clearly not easy.

Makes one wonder how slow it is for the data packets ;) But at say 1500 MTU, 1k per second will be fast enough for most users.
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Online Unixon

  • Frequent Contributor
  • **
  • Posts: 731
I've asked this question to others before, but I usually get answers like "use an RTOS if you have multiple time critical tasks" or "use Linux if you need networking." I understand those points, but I'm still not able to connect them to a real product.
I would put it simpler. You still have to manage resources like time, memory, IO by whatever means.
In bare metal approach this management sits in every piece of code.
So by switching to RTOS you offload this management onto OS and get clean tasks.
And then by switching to even more advanced OS like GNU/Linux you gain access to a huge existing software stack.
My projects were all bare metal up to this time so unfortunately I can't give you a real world example of switching to OS, but the guiding principles are there.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf