How to plan/structure interrupt-based code?

[Siwastaja]

Aerospace has massive budgeting, long timeframes, and large teams consisting of bright minds. They can deal with complexity.

For the rest of us, simplicity should be #1 consideration.

[thinkfat]

Aerospace has massive budgeting, long timeframes, and large teams consisting of bright minds. They can deal with complexity.

For the rest of us, simplicity should be #1 consideration.

Aerospace is also extremely certification-driven. Believe me, they will haggle for every if() in the code because for traceability it will have to have a formal requirement, a justification and a test case.

[Siwastaja]

Yes, and if every if in the code has to be certified, the only solution to keep the development going is to bring on 1000000 lines of code that has passed the paperwork once, maybe decades ago, so that it does not need to be looked at again.

This results in increased complexity and compromised safety, but the process is happy.

Put in enough resources and such disgrace can be made safe enough.

Try to replicate such process in a different setting, thinking it's "robust", and it will be a disaster.

[thinkfat]

The 737MAX disaster is a consequence of this. You can call it greed, but once you understand the cost involved to get an aircraft certified, you kind of understand that Boeing desperately wanted the new aircraft to appear on paper like the model it evolved from.

[JOEBOBSICLE]

Using an RTOS is easy, I don't understand the aversion to them seen here.

If you're trying to cut bread, use a bread knife. If you have multiple independent tasks or multiple interrupts then your RTOS is your bread knife.

You can configure an RTOS to reduce it's flash consumption down to kilobytes. When you start doing global interrupt disables then you're basically doing what an RTOS does but you're potentially disabling interrupts for far too long because you actually need a mutex and not a nuke.

[Simon]

The 737MAX disaster is a consequence of this. You can call it greed, but once you understand the cost involved to get an aircraft certified, you kind of understand that Boeing desperately wanted the new aircraft to appear on paper like the model it evolved from.

That was not even about the code. I would have thought that the code that was new had to be signed off. What they played games with was getting the plane to "look" like the existing version by implementing software that they did not tell the pilots about. The cost saving here was to the airlines who would then place orders because they did not have to retrain the pilots.

I think how this is handled is also about the capabilities of the chip and it's speed. So I swap my 8 bit AVR running at 16MHz for a 32 bit ARM that runs at 48MHz. So to start with it is 3 times the raw speed, secondly it is 32 bit so handles all of those 16 bit variables and operations much, much faster. As I tried to explain it to my MD, we could take an ARM and run it at a fraction of the speed we now run AVR and achieve the same performance at a lower frequency CPU running on a lower voltage and passing EMC more easily.

If you have a slow chip you will want short interrupt routines, even servicing the ADC and PWM means dealing in 16 bit operations on an 8 bit architecture. ARM on the other hand already makes interrupts more long winded because it can with the performance. You don't get an event interrupt on ARM, you get a peripheral interrupt. You then have to check which flag is set. Sounds simple? I wrote some interrupt driven SPI code and fell into several holes before it worked as I worked out how to on the one hand call any of the required bits of code for the various flags whilst ensuring I called the right ones and did not end up with faulty code. The resulting interrupt routine was far from short and I had to think very hard about which is the most used bit of code so that I put flag checks and functions in such an order to make execution as fast as possible most of the time. But you can do this on a chip that is 3 times faster and executes more efficiently.

I think there are two sides to this argument. I sense that the OP just wanted a basic system that had timing for human inputs and outputs. Even with a simple system that runs say every 10ms to sense buttons, turn on LED's or update displays that is fast and even if you missed a cycle a 20ms update to everything is not a problem.

What a lot of people seem to be talking about is mission critical safety applications that of course are entirely different and where you will pay per line of compiled assembly or machine code to have your code audited and as soon as you recompile have to do the same again.

[Howardlong]

Using an RTOS is easy, I don't understand the aversion to them seen here.

If you're trying to cut bread, use a bread knife. If you have multiple independent tasks or multiple interrupts then your RTOS is your bread knife.

You can configure an RTOS to reduce it's flash consumption down to kilobytes. When you start doing global interrupt disables then you're basically doing what an RTOS does but you're potentially disabling interrupts for far too long because you actually need a mutex and not a nuke.

It's about using the right tool for the job. What's the point of delivering a square peg when you have a round hole?

Not all targets have kilobytes of flash.

Some applications are very sensitive about power consumption.

Many applications I deal with have very tight latency and timing requirements, often unsuitable for the generic nature of an RTOS.

Not all applications demand or need an RTOS.

Interrupts and RTOSes aren't the same thing, I'm not sure what that comment was about: if you're not aware of how your system is behaving at that level, RTOS or not, then you have a bigger problem than whether to use an RTOS or not. Using an RTOS doesn't absolve you of needing to write interrupt handlers.

It's horses for courses, and for most of my applications an RTOS is simply unnecessary. If an application includes a GUI, network and/or USB, sure, I can see justification for an RTOS. Most of my projects don't, they're typically functionally simpler and much more deeply embedded. That's not to say others' projects don't typically warrant an RTOS. I just don't see why the OP's project warrants an RTOS, except as an academic exercise.

[JOEBOBSICLE]

If your project starts having multiple interrupts and you don't have an RTOS then you're entering the "assuming variables are atomic" and having no prioritization in place. I have seen this code all over the place as an embedded contractor. Quite frankly a lot of people don't understand thread safety.


If you have tight timing deadlines then you can move interrupts to be higher priority than your normal RTOS level interrupts.
I don't understand why you think using an RTOS would add problems. You can keep doing what you're doing right now but have the convenience of having low priority background tasks too. Having tight timing deadlines is actually quite frankly a reason to use an RTOS.
If whatever you're doing can't afford the cost of a context switch ( last time I measured this it was around 5 microseconds on a 100mhz cortex M4) then I worry that you're doing something in software that should be done in hardware.

The OP should use an RTOS, it really does make sense in 2020.

[Howardlong]

If your project starts having multiple interrupts and you don't have an RTOS then you're entering the "assuming variables are atomic" and having no prioritization in place. I have seen this code all over the place as an embedded contractor. Quite frankly a lot of people don't understand thread safety.


If you have tight timing deadlines then you can move interrupts to be higher priority than your normal RTOS level interrupts.
I don't understand why you think using an RTOS would add problems. You can keep doing what you're doing right now but have the convenience of having low priority background tasks too. Having tight timing deadlines is actually quite frankly a reason to use an RTOS.
If whatever you're doing can't afford the cost of a context switch ( last time I measured this it was around 5 microseconds on a 100mhz cortex M4) then I worry that you're doing something in software that should be done in hardware.

The OP should use an RTOS, it really does make sense in 2020.

Oh dear :-(

5us is pretty pathetic TBH.

But please go right ahead and put RTOSes on your blinkies ;-)

[Simon]

The ultimate decisions around interrupt priority and latency is a feature of the chip and the ability to prioritize one interrupt over another is a feature of the chip not the RTOS. If you chip does not allow you to prioritize interrupts then you have to work closely with the architecture.

[Siwastaja]

JOEBOBSICLE seems to be forcing a square peg into a round hole (while thinking it's the others who are doing that).

5us context switch is totally unacceptable in many applications. An RTOS cannot replace a classical interrupt-based system. It has completely different purpose. RTOSes are for higher-level control systems. Simple bare metal interrupt-based systems are used to control electronics. This usually isn't about having several parallel computation tasks and guaranteeing they get a certain CPU time %. If you are doing that, time-multiplexing calculation in many interrupts, and sharing a lot of data between there threads, you likely want an OS.

But, for example, if you need to build a 3-phase inverter motor controller with current sensing, you definitely do not use an RTOS on the firmware. You initialize the PWM generation timer, calculate FOC algorithm in the timer interrupt, and possibly have another set of interrupts for overcurrent, desaturation, DC bus overvoltage events etc., all of which can be triggered in some hundreds nanoseconds, not 5µs.

Sure, both approaches can coexist (a more sophisticated inverter could use an RTOS and interrupts, or even opt for two-chip solution where another microcontroller runs bare metal and another runs a full-fledged OS, RT or not), but then you need to understand and be able to do both. If the total complexity doesn't require an RTOS, then it's just extra weight. This would be the OP's case.

I also fail to see how an RTOS magically makes variables atomic. The programmer needs to remember to use specific atomic constructs, just like in bare-metal.

All ARM Cortex microcontrollers implement interrupt priorities; it's very simple to use; one LoC of initialization, and one LoC to set the priority for the interrupt.

Finally, as JOE himself says, "Quite frankly a lot of people don't understand thread safety." I wholeheartedly agree. The problem doesn't go away with RTOS; quite the opposite, much of the problem is created by the OS, you bring in mutexes, possibilities for deadlocks, thread synchronization, all that pile of complicated shit many of us fail at. Of course, the problem originates from the hardware (and the OS abstracts it to look different) so not using an OS isn't a magical solution either, but a simple interrupt-based bare-metal system is easier to understand as long as it really stays simple enough not to warrant the added abstraction. This would be the case for the OP.

The idea of something being "modern" and having to be used only because of that is dangerous. Use the right tool for the job.

[dietert1]

There is a difference being a specialist and publish an RTOS framework and being a specialist writing smart comments in this forum. Especially for a beginner in embedded trying to understand one of the published frameworks is the recommended approach. All those smart comments here won't get him/her anywhere. A stupid idea to think one can do those things better than the specialists who spent their whole career thinking about concurrency, thread safety, priorities and the like.

Regards, Dieter

[Simon]

There is a difference being a specialist and publish an RTOS framework and being a specialist writing smart comments in this forum. Especially for a beginner in embedded trying to understand one of the published frameworks is the recommended approach. All those smart comments here won't get him/her anywhere. A stupid idea to think one can do those things better than the specialists who spent their whole career thinking about concurrency, thread safety, priorities and the like.

Regards, Dieter

The sad thing is that this thread has simply been a chest beating match way out of the scope of what the OP asked for. They asked to do something I myself pondered at one time and have only just started to do. It's not complicated but that important first step out of code that runs dumbly in a loop making poor use of the interrupt resources to something that can handle things in a semiautomatic way. Not hard, I have done it a couple of times myself.

Instead of some help on first steps they have gotten cautionary advice fit only for people trying to design something like a bloody ballistic missile system..... What he wants to do is simple, it's a question I once asked our subcontractor at work about how he did his. Once he explained the basic principles I was able to get something up and running.

[Jan Audio]

Indeed you all scared the new member.
Curious if his PSU is working...
Always fun these topics.

[Siwastaja]

It never stops amusing me how certain people can't resist posting "this thread is crap with no content" comments, do they think their such comments lead the way? It's a handful of people, they never seem to give meaningful contributions. Oftentimes, there is some good discussion if you give it a chance, it's just that the people with limited attention span seem to think that all the others have the same attention span problem as well and will "run away" if they see a message longer than 100 words, let alone something that considers different viewpoints instead of directly answerring a question. These people never learn anything, just keep complaining everything is TLDR or "offtopic" even when it isn't.

If you have nothing to say, just do not post. Lead the way; if you feel the content is of wrong type, provide the right content type, instead of complaining about it.

[NorthGuy]

Indeed you all scared the new member.
Curious if his PSU is working...
Always fun these topics.

In the beginning, the OP seemed to want to take every advise given. Perhaps now he realized the futility of this idea and started thinking on his own. If this is the case, this thread was a great success.

[Simon]

Indeed you all scared the new member.
Curious if his PSU is working...
Always fun these topics.

In the beginning, the OP seemed to want to take every advise given. Perhaps now he realized the futility of this idea and started thinking on his own. If this is the case, this thread was a great success.

Yes and no, what I would have learnt is don't bother asking as you will be told several ridiculously complicated ways of doing it and warned about the end of the world when nothing like that will happen.

[Siwastaja]

I agree that OP's case is quite simple. Run some UI code / command parsing / reporting in main while(1). If you have some PI(D) feedback loops for voltage/current, run them in the ADC conversion finished ISR. If you have safety events, run the associated shutdown signals in high-priority interrupts. There are some traps regarding synchronization and shared state: for example, if your main code enables power, and your safety ISR disables the power, need to make sure "enable power" commands are ignored (for some time? until acknowledged?) in the main code, and that there is no short time window where this ignoring won't work. You need to think about it, but it's not going to be a big thing.

The best way to learn is to pick some method and implement it using it. Such simple project can succeed in many different ways. Then you get the first-hand experience and can think if you could do it otherwise.

[Simon]

Precisely, trying to write something that first time is fit for a missile control system is doomed to fail.

[thinkfat]

If the OP reads this thread carefully, and is able to think for himself (which I assume), he'll find a lot of useful information and potential traps and pitfalls in here. He'll also find that, even here, nothing that is written on the Internet is to be taken at face value

[Fixpoint]

Systems often grow so large and complex that no person could ever have an understanding of the whole system.

That is true. Therefore it is necessary to do something about the complexity. Simplicity has always been a hallmark of good engineering.

Actually, that is a ridiculous myth. Just have a look at real-world systems like an oscilloscope or an RF spectrum analyzer. The electrical engineering is complicated and requires years of education and much intelligence. Many people cannot even design a multimeter themselves because they will soon discover that the "simple" principles are very hard in practice. And mechanical or chemical engineering? Same story. How "simple" is it to build a rocket?

And now, please try to build a self-stabilizing quad copter yourself, including designing and implementing the software and the differential equations. Have fun with your "simplicity".

"Fools ignore complexity, professionals deal with complexity, geniuses remove complexity."

Well, that is only half of the picture. There are two different kinds of complexity, namely intrinsic complexity and accidental complexity. You can only take away accidental complexity, but not intrinsic complexity. That's the one you simply have to deal with, and if you can't, you'll create a mess.

Yet, the modern computer science believes that the way to manage complexity is to add more complexity [...] The modern computer science took a wrong turn. Instead of researching how to improve quality of the software being build,

It doesn't seem that you know what you are talking about here, neither regarding the theoretical side of CS nor the practical side.

the computer science concentrates on the programming process itself and try to create better rules and mechanisms.

No, it doesn't. Just no. Just have a look into CS what people are doing there. BTW, programming isn't CS, in case that's what you're thinking, just like laying bricks isn't civil engineering.

CS creates neither complexity nor simplicity. CS is concerned with developing mathematical models and mathematical tools for *dealing with* complexity *in general*. It is up to you whether you want to ignore this or not, but that's how it is.

Software is getting bloated beyond comprehension, buggy, and unmanageable.

Yes. That's the result of bad engineering, i.e. misled product managament and bad software development. You are mixing up CS and development. What you are doing is like saying "mathematics took a wrong turn" when your circuit isn't working. No, that's not the result of bad mathematics but of bad engineering.

[Siwastaja]

Having the mindset of achieving simplicity is exactly important because many problems are inherently so complex.

They are complex even if you do them the simplest way possible. Such projects have a real risk of blowing up in complexity until they become nearly impossible to finish unless the team understands how to simplify.

For example, a self-stabilizing quad-copter optimally requires the workload of about 4-5 (technical engineer) team members and about a year or two of development time. But getting to this requires the mindset of simplification, understanding which parts are necessary which are not. You can easily get lost by applying modern software industry practices, for example, and you definitely can hire 200 trained monkeys fresh out of CS class to design a middleware for your quadcopter communication bus. This is not unseen; such projects just don't ever work out so the end consumers won't hear about them, but those "in the industry" will.

So being able to simplify will be one of the major deciding factors whether the project will be successful or not. The exception are fundamentally very simple projects which can finally succeed even with a lot of bloat. Many modern information technology services are such examples; think about selling a train ticket online, something a hacker write over a weekend, and a large software house can spend years and tens of millions to pull off badly.

[Fixpoint]

You can easily get lost by applying modern software industry practices, for example, and you definitely can hire 200 trained monkeys fresh out of CS class to design a middleware for your quadcopter communication bus.

Yes. But that's not the point. The point is that we understand that this has nothing to do with computer science. What you are describing is a (rather classic) management error. Someone made a wrong management decision for whatever reason -- there are very many very different reasons why someone could act like that. Anyway, CS is not concerned with business administration. Also, it is not concerned with the question whether you should design a communication bus at all.

In simple terms (we want it simple, right?), computer science is concerned with

(a) algorithms,
(b) data structures,
(c) both formal and semi-formal meta-models thereof.

For example, software development is an application of all of that.

think about selling a train ticket online, something a hacker write over a weekend

  No. Just think for a minute. The interfacing with all the third-party systems ALONE will take much time and many people. And you didn't even design the routing algorithms etc. Let alone the user interface. Etc etc etc. It's a very complex system.

[NorthGuy]

How "simple" is it to build a rocket?

If a rocket is more complex than a LED blinker is not a reason to make the rocket more complex as it already is. Nor is it a reason to make LED blinker as complex as a rocket.

What you are doing is like saying "mathematics took a wrong turn" ...

Mathematics can take a wrong turn too. If you destroy the underlying principles, for example if you decide that no proof is needed for theorems, the mathematics will become completely useless within few years. We have seen similar things in the past. After the destruction of Roman empire, the whole science got derailed and moved nearly nowhere for over a thousand years.

Of course, the mathematical parts of the computer science, such as sorting algorithms, are Ok, they existed long ago. But the computer science also deals with computer languages, programming paradigms, general methods and more. For example, why do you think RTOS is necessary in all your projects? You probably went to a college and you were taught this in one (or many) of your courses. Did they give you a mathematical proof of that? No? Then this has nothing to do with mathematics, but still a part of the computer science. Does this make you a bad engineer? Of course not.

[Fixpoint]

Mathematics can take a wrong turn too. If you destroy the underlying principles, for example if you decide that no proof is needed for theorems, the mathematics will become completely useless within few years. We have seen similar things in the past. After the destruction of Roman empire, the whole science got derailed and moved nearly nowhere for over a thousand years.

Um ... if we take away the proofs, then it's just not mathematics anymore. That would not be a "wrong turn" but just nonsense.

But the computer science also deals with computer languages, programming paradigms, general methods and more.

Well, yes, CS "deals" with that, but let's be more precise: In computer science, we don't *create* programming languages but *study* them. CS is not an engineering discipline, so nothing is built there, only studied.

CS develops principles that are fundamental *for* programming languages. There are some computer scientists (persons!) who create programming languages, but don't confuse those guys with the academic discipline *itself*.

There are research programming languages (like Prolog or many functional languages, for instance) that were created at universities. They were designed for study, not for practical use. In practice, research languages are of limited use, and that's ok, because that's not what they are made for.

For example, why do you think RTOS is necessary in all your projects?

I don't and I never said that. What I say is that IF you need a scheduler, THEN it should be preemptive. But if you don't need a scheduler, then so be it. In general, you do NOT need to use an RTOS.

you were taught this in one (or many) of your courses.

I was never taught anything about RTOS, I learned about that the hard way -- on the fly in projects. My opinions don't come from the lecture halls but from my experience in real-world projects.