Embedded Rust on microcontrollers (Cortex M, Cortex A, Softcores etc.)

[coppice]

Recently I have been looking at Lua and Micropython to implement higher level logic in embedded projects. A project I'm currently working on will very likely have the high level logic / functionality implemented in Micropython. Besides having code run inside a VM, one of the other advantages is that the code can be compiled on the target itself (as part of the initialisation) so it is easier to replace the logic that implements the functionality of the device without needing to recompile the entire source.

Lua doesn't seem to get the coverage it deserves. I haven't done much with it, but I've seen people who do. Once they start using it they don't usually go back. It seems to work well for them.

[NorthGuy]

In the context of MCU firmware, despite some marginal ugliness, C is a robust and proven tool, possibly even the best, but it's not THE definitive tool, there is always space for some kind of progress. I'm not saying that Rust will necessarily be a progress but, having time to waste (my definition of hobby), why not keep an open mind and give it a look?

There's progress. Newer MCUs are getting better periphery, the peripheral modules get interconnected in various ways, more and more tasks can be off-loaded to peripheral, you get more memory so you can create better algorithms, you get faster and more reliable communications. That's where the progress is, not in languages.

Let me ask you a question. If you want to create a software program, you need to design data structures, algorithms, processes. When you do that, do you ever need to know what language you're going to use to implement it?

[tggzzz]

In the context of MCU firmware, despite some marginal ugliness, C is a robust and proven tool, possibly even the best, but it's not THE definitive tool, there is always space for some kind of progress. I'm not saying that Rust will necessarily be a progress but, having time to waste (my definition of hobby), why not keep an open mind and give it a look?

There's progress. Newer MCUs are getting better periphery, the peripheral modules get interconnected in various ways, more and more tasks can be off-loaded to peripheral, you get more memory so you can create better algorithms, you get faster and more reliable communications. That's where the progress is, not in languages.

Such parallelism implies concepts and thinking that don't come naturally to most people coming from a pure software background. Parallelism and distributed systems are not as well taught as they should be. People coming from a digital hardware background don't have much trouble with those concepts.

That progress in hardware needs to be matched by progress in languages. Sticking plasters aren't sufficient.

Let me ask you a question. If you want to create a software program, you need to design data structures, algorithms, processes. When you do that, do you ever need to know what language you're going to use to implement it?

Yes, you do, since the language affects what you can and cannot express. There are similar issues with human languages.

Having said that, many languages are more or less equivalent in the sense that if you grok one then switching to another is not a big hurdle, e.g.C/C++/Delphi, or Smalltalk/Java/C#, or VHDL/Verilog, or Erlang/xC/Occam, or SQL, or Matlab/Mathematica, or Prolog.

Example: you wouldn't suggest using SQL to send messages from one Unix box to another, would you.

[coppice]

Such parallelism implies concepts and thinking that don't come naturally to most people coming from a pure software background. Parallelism and distributed systems are not as well taught as they should be. People coming from a digital hardware background don't have much trouble with those concepts.

That progress in hardware needs to be matched by progress in languages. Sticking plasters aren't sufficient.

The problem with temporal issues in systems is most people are very bad at getting their heads around them. I don't see how any kind of tool less than a machine which can totally replace a human will ever do much about that.

[tggzzz]

Such parallelism implies concepts and thinking that don't come naturally to most people coming from a pure software background. Parallelism and distributed systems are not as well taught as they should be. People coming from a digital hardware background don't have much trouble with those concepts.

That progress in hardware needs to be matched by progress in languages. Sticking plasters aren't sufficient.

The problem with temporal issues in systems is most people are very bad at getting their heads around them. I don't see how any kind of tool less than a machine which can totally replace a human will ever do much about that.

I think hope that is a bit strong!

Most of the current tools contain features that actively sabotage being able to reliably express parallelism. That has to stop.

It will stop, but only over the course of a generation and with adherents of the current tools kicking and screaming all the way. That is the same way that scientific dogma atrophies.

[coppice]

Such parallelism implies concepts and thinking that don't come naturally to most people coming from a pure software background. Parallelism and distributed systems are not as well taught as they should be. People coming from a digital hardware background don't have much trouble with those concepts.

That progress in hardware needs to be matched by progress in languages. Sticking plasters aren't sufficient.

The problem with temporal issues in systems is most people are very bad at getting their heads around them. I don't see how any kind of tool less than a machine which can totally replace a human will ever do much about that.

I think hope that is a bit strong!

Most of the current tools contain features that actively sabotage being able to reliably express parallelism. That has to stop.

It will stop, but only over the course of a generation and with adherents of the current tools kicking and screaming all the way. That is the same way that scientific dogma atrophies.

I'm not sure its the tools that really make things hard. A lot of it is poor documentation. When you see the potential gotchas, and look up exactly what is going on behind the scenes, too often you can't find the answer without massive effort.

If this will stop over a generation, why is the problem still here? We've been faffing around with this since the late 70s.

[nctnico]

Anyway I'm digressing, I'm writing this to thank you all from preventing me waste my time. I still hope that somewhere lies a better language that, even if not solving problems for me, can make my programming experience less frustrating but again, (to use Mario's words) our princess is in another castle... 

Recently I have been looking at Lua and Micropython to implement higher level logic in embedded projects. A project I'm currently working on will very likely have the high level logic / functionality implemented in Micropython. Besides having code run inside a VM, one of the other advantages is that the code can be compiled on the target itself (as part of the initialisation) so it is easier to replace the logic that implements the functionality of the device without needing to recompile the entire source.

That would be a *completely* different story. Good only when the ratio between computational resources (therefore cost?) to problem complexity is high.

No. You can run embedded versions of Lua (not specifically eLua) on a standard microcontrollers. You'd need about 128kB of flash and 32kB of RAM to run a Lua based application. Last year I've been working on a Lua 'fork' that can be used without an OS as part of a round-robin scheduler, non-blocking process. I have not got round to finishing it yet. Maybe this summer holiday...

[tggzzz]

Such parallelism implies concepts and thinking that don't come naturally to most people coming from a pure software background. Parallelism and distributed systems are not as well taught as they should be. People coming from a digital hardware background don't have much trouble with those concepts.

That progress in hardware needs to be matched by progress in languages. Sticking plasters aren't sufficient.

The problem with temporal issues in systems is most people are very bad at getting their heads around them. I don't see how any kind of tool less than a machine which can totally replace a human will ever do much about that.

I think hope that is a bit strong!

Most of the current tools contain features that actively sabotage being able to reliably express parallelism. That has to stop.

It will stop, but only over the course of a generation and with adherents of the current tools kicking and screaming all the way. That is the same way that scientific dogma atrophies.

I'm not sure its the tools that really make things hard. A lot of it is poor documentation. When you see the potential gotchas, and look up exactly what is going on behind the scenes, too often you can't find the answer without massive effort.

You need the right conceptual primitives and concepts. Sticking plasters added later don't count!

You need simplicity and economy of concepts. Complexity is the enemy of comprehension and prediction.

If this will stop over a generation, why is the problem still here? We've been faffing around with this since the late 70s.

It is the usual two steps forward one step backward. Meanwhile semiconductor process scaling was making ten steps forward. That's ceased, so people are having to become more imaginative and inventive. Plus the new generation starts with parallel distributed system.I

[SiliconWizard]

I like Lua as a scripting language, but any "embedded" version I've seen is non-official, usually doesn't support the whole Lua language (for obvious reasons), and most are not well or actively maintained.
Happy to be proven wrong though.

One variant I've mentioned at some point was nelua. Heavily inspired by Lua, but with a whole lot more, with things more adapted to embedded software.
It compiles to C, so should be usable with pretty much any target with some minimal preliminary work.
https://nelua.io/
Of course it's a compiled language, not an interpreted one, so it doesn't quite fit the same purpose or use case. But as a replacement for C for those who are curious and willing to experiment, why not.

[coppice]

I like Lua as a scripting language, but any "embedded" version I've seen is non-official, usually doesn't support the whole Lua language (for obvious reasons), and most are not well or actively maintained.

A key thing about embedded stuff is it doesn't have to be portable. Being "official" has few benefits.

[nctnico]

I like Lua as a scripting language, but any "embedded" version I've seen is non-official, usually doesn't support the whole Lua language (for obvious reasons), and most are not well or actively maintained.

Lua is mature; it doesn't need to be maintained. It ain't broken, so it doesn't need fixing / adding more bugs. For my own fork, I started with this: https://github.com/szieke/embLua.

[SiliconWizard]

I like Lua as a scripting language, but any "embedded" version I've seen is non-official, usually doesn't support the whole Lua language (for obvious reasons), and most are not well or actively maintained.

Lua is mature; it doesn't need to be maintained. It ain't broken, so it doesn't need fixing / adding more bugs.

Yes, sure. That's exactly what I said to my grandma the other day.

[uliano]

There's progress [...], not in languages.

You can't be serious 

[uliano]

That would be a *completely* different story. Good only when the ratio between computational resources (therefore cost?) to problem complexity is high.

No. You can run embedded versions of Lua (not specifically eLua) on a standard microcontrollers. You'd need about 128kB of flash and 32kB of RAM to run a Lua based application. Last year I've been working on a Lua 'fork' that can be used without an OS as part of a round-robin scheduler, non-blocking process. I have not got round to finishing it yet. Maybe this summer holiday...

The "ratio" thing is nice because it also has a denominator. I'm sure eLua is really nice and well fit for many applications, but it won't go on par with compiled code in terms of resources (flash, ram and also time).

[nctnico]

That would be a *completely* different story. Good only when the ratio between computational resources (therefore cost?) to problem complexity is high.

No. You can run embedded versions of Lua (not specifically eLua) on a standard microcontrollers. You'd need about 128kB of flash and 32kB of RAM to run a Lua based application. Last year I've been working on a Lua 'fork' that can be used without an OS as part of a round-robin scheduler, non-blocking process. I have not got round to finishing it yet. Maybe this summer holiday...

The "ratio" thing is nice because it also has a denominator. I'm sure eLua is really nice and well fit for many applications, but it won't go on par with compiled code in terms of resources (flash, ram and also time).

You should also think about things like reliability, robustness and focussing on implementing a solution. As I wrote earlier in this thread, writing robust C code is rather tedious because you'll need to prevent range & buffer overflows. This draws attention away from dealing with the actual programming problem at hand. With languages like Python and Lua you can't have problems with buffer overflows and pointers going wrong. And if something bad happens, you can use exceptions to recover gracefully. On top of that, Python and Lua are way more suitable for dealing with data sets and text / strings which reduces development time while producing a higher quality result (as in easy to read, to the point code).

[Siwastaja]

You should also think about things like reliability, robustness and focussing on a problem. As I wrote earlier in this thread, writing robust C code is rather tedious because you'll need to prevent range & buffer overflows. This draws attention away from dealing with the actual programming problem at hand.

I don't agree, at all. The problem with C is not that you have to do input validation, it's that the consequences of not doing it are more dire than it needs to be.

Writing in a language which does not corrupt memory and trusting the language instead of validating inputs results in code that does not crash in the most strict meaning of word, but still does not work and fails in almost as badly. Very typical example is some unhandled exception propagate gazillion levels and then just get good old crash (from user's viewpoint) with ten pages of cryptic exception stack trace (and in case of embedded, no way to propagate any of this to the user or developer).

Instead, if one checks the inputs and returns -1, and the caller handles this error condition, then it can (a) report the issue, (b) use some fail-safe logic.

This is a textbook example of false sense of security. Input validation is always required and there is no silver bullet; it's hard work, because it involves design; designing how errors are functionally handled. Not corrupting memory is not enough.

[tggzzz]

You should also think about things like reliability, robustness and focussing on a problem. As I wrote earlier in this thread, writing robust C code is rather tedious because you'll need to prevent range & buffer overflows. This draws attention away from dealing with the actual programming problem at hand.

Precisely, but plus more practical pitfalls and distractions, of course. Anything that reduces the distractions is to be welcomed, provided it doesn't bring other problems.

Based on the practical evidence, I'd change "rather tedious" to "rather tedious and error prone".

[nctnico]

@Siwastaja: You are missing the point. For example: in Python and Lua you can have dynamically sized arrays / data sets out of the box. In C you either need to know the maximum size at design time OR use some kind of library that implements that for you. Both are extra work and sources for errors (for example: getting the maximum size wrong or being different at various places in software). Writing robust software is not just about input validation but also about minimizing the potential for programming errors.

[Siwastaja]

@Siwastaja: You are missing the point. For example: in Python and Lua you can have dynamically sized arrays / data sets out of the box. In C you either need to know the maximum size at design time OR use some kind of library that implements that for you. Both are extra work and sources for errors (for example: getting the maximum size wrong or being different at various places in software). Writing robust software is not just about input validation but also about minimizing the potential for programming errors.

I don't disagree with that, so yeah, I probably missed your point. Yes, with C you need to do more, which means more surface area for bugs, too.

Languages which already solve problems for you still don't free you from the same type of input validation and range checking you do in C; just you don't need to do that for things that already work out of box, of course. You can use foreach to iterate a table, but if you need X elements for your algorithm to operate correctly, you still need to check you have those.

[uliano]

That would be a *completely* different story. Good only when the ratio between computational resources (therefore cost?) to problem complexity is high.

No. You can run embedded versions of Lua (not specifically eLua) on a standard microcontrollers. You'd need about 128kB of flash and 32kB of RAM to run a Lua based application. Last year I've been working on a Lua 'fork' that can be used without an OS as part of a round-robin scheduler, non-blocking process. I have not got round to finishing it yet. Maybe this summer holiday...

The "ratio" thing is nice because it also has a denominator. I'm sure eLua is really nice and well fit for many applications, but it won't go on par with compiled code in terms of resources (flash, ram and also time).

You should also think about things like reliability, robustness and focussing on implementing a solution. As I wrote earlier in this thread, writing robust C code is rather tedious because you'll need to prevent range & buffer overflows. This draws attention away from dealing with the actual programming problem at hand. With languages like Python and Lua you can't have problems with buffer overflows and pointers going wrong. And if something bad happens, you can use exceptions to recover gracefully. On top of that, Python and Lua are way more suitable for dealing with data sets and text / strings which reduces development time while producing a higher quality result (as in easy to read, to the point code).

You don't have to tell me this, as a matter of fact in my real job I'm using Python since maybe 20 years [I also paid (as maybe other 20-25 people in the world) for the documentation of numpy when it was not free as the code itself :-D ]. In those times the huge library that makes Python so useful today wasn't there and I had to write some function in C to speed up my image processing application. So I KNOW that there ARE overheads. The huge python library itself it is not even written in Python and probably neither or mostly-not available for micropython so you are just moving your "ghosts" to the people who wrote the code for you.

This double language problem is well known and attempts were made to overcome it as e.g. in Julia, alas not a very successful attempt in my opinion.

[NorthGuy]

@Siwastaja: You are missing the point. For example: in Python and Lua you can have dynamically sized arrays / data sets out of the box. In C you either need to know the maximum size at design time OR use some kind of library that implements that for you. Both are extra work and sources for errors (for example: getting the maximum size wrong or being different at various places in software). Writing robust software is not just about input validation but also about minimizing the potential for programming errors.

You can use dynamic allocation in C too. malloc() is, sort of, built-in. Whether or not you should use dynamic memory allocations is another question. It depends on the project. This is totally different story.

If you don't like malloc(), you can write your own allocator in C, even in assembler. This is actually very easy to do. Can you write your own allocator in Python?

Of course, allocation may fail if you're running out of memory. Have you tested what happens to your Python programs if Python itself runs out of memory? How would it affect your customers?

[nctnico]

@Siwastaja: You are missing the point. For example: in Python and Lua you can have dynamically sized arrays / data sets out of the box. In C you either need to know the maximum size at design time OR use some kind of library that implements that for you. Both are extra work and sources for errors (for example: getting the maximum size wrong or being different at various places in software). Writing robust software is not just about input validation but also about minimizing the potential for programming errors.

You can use dynamic allocation in C too. malloc() is, sort of, built-in. Whether or not you should use dynamic memory allocations is another question. It depends on the project. This is totally different story.

You are also missing the point. You really shouldn't need or want to be messing with dynamic memory allocation at all! Let well tested & proven libraries deal with that deep under the hood somewhere. When I need dynamically allocated structures in C, I will use C++ and STL templates that do this for me so the room for mistakes is next to non-existent. I've had my fair share of debugging time with valgrind to find other people's memory allocation mistakes. I don't want to add to that mess.

Recently I put solar panels on my roof. If you look at the 'professional' installers, 99% of them walk on sloped roofs without any safety measures. I, however, used a certified safety harness and attachements while working on the roof. Bottom line: just because something is possible or seems to be the standard way of doing something, it doesn't mean it is a wise thing to do. Safety first.

[nctnico]

That would be a *completely* different story. Good only when the ratio between computational resources (therefore cost?) to problem complexity is high.

No. You can run embedded versions of Lua (not specifically eLua) on a standard microcontrollers. You'd need about 128kB of flash and 32kB of RAM to run a Lua based application. Last year I've been working on a Lua 'fork' that can be used without an OS as part of a round-robin scheduler, non-blocking process. I have not got round to finishing it yet. Maybe this summer holiday...

The "ratio" thing is nice because it also has a denominator. I'm sure eLua is really nice and well fit for many applications, but it won't go on par with compiled code in terms of resources (flash, ram and also time).

You should also think about things like reliability, robustness and focussing on implementing a solution. As I wrote earlier in this thread, writing robust C code is rather tedious because you'll need to prevent range & buffer overflows. This draws attention away from dealing with the actual programming problem at hand. With languages like Python and Lua you can't have problems with buffer overflows and pointers going wrong. And if something bad happens, you can use exceptions to recover gracefully. On top of that, Python and Lua are way more suitable for dealing with data sets and text / strings which reduces development time while producing a higher quality result (as in easy to read, to the point code).

You don't have to tell me this, as a matter of fact in my real job I'm using Python since maybe 20 years [I also paid (as maybe other 20-25 people in the world) for the documentation of numpy when it was not free as the code itself :-D ]. In those times the huge library that makes Python so useful today wasn't there and I had to write some function in C to speed up my image processing application. So I KNOW that there ARE overheads. The huge python library itself it is not even written in Python and probably neither or mostly-not available for micropython so you are just moving your "ghosts" to the people who wrote the code for you.

It is not about moving ghosts, but using well tested & proven libraries / existing code in a way that makes it hard to add mistakes. Python and Lua (for example, not saying you should limit yourself to Python or Lua) allow that.

[westfw]

in Python and Lua you can have dynamically sized arrays data sets out of the box.

Just out of curiosity, what happens to the average Python or Lua embedded program that has used "out of the box dynamically sized arrays" if those arrays exceed available memory?

I guess it probably throws some sort of exception that the programmer probably didn't plan for, causing ... something...  (Shades of Ariane ?)  (Not that  carefully checking the return value of every malloc() in C (or any other language) means that you have a obvious and effective course of action, either.)

[nctnico]

in Python and Lua you can have dynamically sized arrays data sets out of the box.

Just out of curiosity, what happens to the average Python or Lua embedded program that has used "out of the box dynamically sized arrays" if those arrays exceed available memory?

I guess it probably throws some sort of exception that the programmer probably didn't plan for, causing ... something...  (Shades of Ariane ?)  (Not that  carefully checking the return value of every malloc() in C (or any other language) means that you have a obvious and effective course of action, either.)

The 'C side' can catch that because the Python / Lua VM exits and then can decide a reset or some other form of recovery. This is contained within a few lines of C code. It doesn't require to propagate an out-of-memory error many layers up to some application layer which may not even have a way to deal with such an error. Ofcourse the error can also be handled inside the Lua / Python program using an exception handler.