Lightweight Scripting Language for MCUs

[helius]

Yes, RPL is very similar to FORTH but also with some interesting differences.

RPL is dynamically typed, and knows the type of every object. Forth is untyped, and doesn't know the type of any object. That is such a large difference that the claimed similarity seems overstated.

RPL also is able to assign (and remember) a label attached to any object, as well as units for number objects. Not many other languages have that capability (I believe one might be Clojure?)

The absence of comments in stored programs is expected, based on the fact that the programs are stored in parsed form (as opposed to ASCII text). This is a problem that also arose in Lisp 1.5 and INTERLISP, and the approach there was to use a (COMMENT ...) form that was defined to do nothing. Since RPL is postfix, there is no simple way to achieve that, although you could simply write "Comment..." DROP to nullify the stack effect of the comment.

[audiotubes]

They're very similar in that they both use a stack-based paradigm, and that code is written by building up solutions from (ideally small) words that perform a specific function.

[brucehoult]

The absence of comments in stored programs is expected, based on the fact that the programs are stored in parsed form (as opposed to ASCII text). This is a problem that also arose in Lisp 1.5 and INTERLISP, and the approach there was to use a (COMMENT ...) form that was defined to do nothing. Since RPL is postfix, there is no simple way to achieve that, although you could simply write "Comment..." DROP to nullify the stack effect of the comment.

But isn't that *precisely* a simple way to do exactly the same thing?

Or do ...

Code: [Select]

« DROP » → COMMENT «
  "get qty" COMMENT
  "Qty:" PROMPT
  "get unit price" COMMENT
  "Price:" PROMPT
  "display total" COMMENT
  → Q P « "Total " Q P * →STR + MSGBOX »
»

... and be happy.

[helius]

It certainly has the same effect. But (unlike program syntax), comments' only purpose is to be read by humans, and the postfix construction is not easy to read. Even Forth uses parenthesized comments, set off before and after by easily identified brackets.

I should clarify that the INTERLISP style of comment is no longer used in Lisp code since around 40 years ago. It was a kludge based on the problem of programs being stored and edited in a parsed form, which was abandoned as the Common Lisp standard displaced older dialects.

[gnuarm]

It certainly has the same effect. But (unlike program syntax), comments' only purpose is to be read by humans, and the postfix construction is not easy to read. Even Forth uses parenthesized comments, set off before and after by easily identified brackets.

In Forth, it's much more common to denote comments by a backslash, with the comment ending at EOL

\ On test failure, set output frequency
: Test20kHz ( -- )  20000 GetAmpMeas . ;
CR .( Done loading Test20kHz)

The last line was part of the debugging code I added to isolate a problem.

It's not postfix that is hard to read, it is poorly written programs in any form that are hard to read.   One thing that is important in Forth, which is not so important in other languages, is to write code that is clear and simple.  Other languages allow, or even promote poor program design.

[brucehoult]

comments' only purpose is to be read by humans, and the postfix construction is not easy to read.

I would never dream of saying such a thing!! I've used postfix languages a lot (mostly PostScript). But ok ....

[Smokey]

What is the state of things with embedded systems scripting in 2024?

My requirements are actually pretty minimal.  I have a fully functional ESP32 system (ESP-IDF, so FreeRTOS and all that) to which I want to embed a user scripting engine so the user can make their own simple LED animations.  It needs to be setup so they don't need to externally compile and flash anything (which they don't actually have access to do anyway).  Anyone know of something that would solve that problem?

[tggzzz]

What is the state of things with embedded systems scripting in 2024?

My requirements are actually pretty minimal.  I have a fully functional ESP32 system (ESP-IDF, so FreeRTOS and all that) to which I want to embed a user scripting engine so the user can make their own simple LED animations.  It needs to be setup so they don't need to externally compile and flash anything (which they don't actually have access to do anyway).  Anyone know of something that would solve that problem?

Forth.

[nctnico]

What is the state of things with embedded systems scripting in 2024?

My requirements are actually pretty minimal.  I have a fully functional ESP32 system (ESP-IDF, so FreeRTOS and all that) to which I want to embed a user scripting engine so the user can make their own simple LED animations.  It needs to be setup so they don't need to externally compile and flash anything (which they don't actually have access to do anyway).  Anyone know of something that would solve that problem?

I've used Micropython for a similar situation. Embedded Lua (emblua, not eLua!) could be more suitable though as this is easier to integrate but in my case the customer insisted on using Python. Micropython needed a bit of hacking to integrate it into a C based ESP32 project and use the flash filesystem to read/store a script. The ESP32 IDF environment offers a Posix file I/O layer so Micropython has to be told it runs on a Unix system but still use a defined heap space.

[SiliconWizard]

Forth and Lua. Could also be a minimal, ad hoc scripting "language" if neither of the existing solutions are small enough to fit in said MCU.
The "MCU" is a hard requirement here.
What you can fit in a ESP32, or otherwise some other similar MCU with (relatively) a lot of Flash memory and RAM, is one thing, what would fit in a MCU with a lot less memory available is another entirely.

[nctnico]

Forth and Lua. Could also be a minimal, ad hoc scripting "language" if neither of the existing solutions are small enough to fit in said MCU.
The "MCU" is a hard requirement here.
What you can fit in a ESP32, or otherwise some other similar MCU with (relatively) a lot of Flash memory and RAM, is one thing, what would fit in a MCU with a lot less memory available is another entirely.

For embLua you mostly need flash. Like 128kB to be on the safe side. A simple Lua script needs a few kB of RAM at most. IIRC I have done tests using an LPC1225 which has 8kB of RAM. For Micropython 32kB of RAM is a good start.

[Smokey]

https://github.com/szieke/embLua

embLua requirements
To run scripts which do something 'meaningful' embLua needs:

RAM: 1k Stack, 11k Heap
ROM (not optimized code): 120 on MSP430, 90k on Renesas
C-Library

[Picuino]

EDIT:
Has anything been said about TCL yet?
Does anyone know it as a scripting language for microcontrollers?

https://wiki.tcl-lang.org/page/Small+Tcl
https://tinytcl.sourceforge.net/

[tellurium]

If the anticipated code is trivial, one can consider very, very stripped down Javascript:

https://github.com/cesanta/elk/

And yeah, TCL!
Another tiny implementation is here:
https://github.com/zserge/partcl (code)
https://zserge.com/posts/tcl-interpreter/ (article)

[coppice]

EDIT:
Has anything been said about TCL yet?
Does anyone know it as a scripting language for microcontrollers?

https://wiki.tcl-lang.org/page/Small+Tcl
https://tinytcl.sourceforge.net/

I would avoid TCL, not because of anything inherently wrong with it, but because it has slumped from a prominent mainstream position to obscurity. Everything I know that is written in TCL is something old that is being maintained, but almost certainly wouldn't be started afresh in TCL. Its frequently hard to get people to put the effort into even simple maintenance of TCL code, which is often really sad. When a language gets in that state you aren't going to get much buy in from others. You are on your own.

[Smokey]

It seems like embLua is a clear winner at this point (memory footprint, functionality, language complexity, etc), at least for embedding scripting into an already existing codebase like I need.  Am I wrong?

[coppice]

It seems like embLua is a clear winner at this point (memory footprint, functionality, language complexity, etc), at least for embedding scripting into an already existing codebase like I need.  Am I wrong?

A lot of people who use it stick with it, so it seems to be a solution that results in reasonable levels of contentment.

[ksjh]

A while ago, I was looking for compact ESP32-C3 boards on AliExpress and found some marked "LuatOS-Core", so I tried to find out more about those boards and stumbled upon https://wiki.luatos.org/ when trying to find the pinout of those boards. I did not try LuatOS at all or look at it in more detail, but I thought it could be useful for your project. There seem to be implementations for the ESP32-C3 and the ESP32-S3, source is available on gitee, I did not check the licensing.

[nctnico]

A while ago, I was looking for compact ESP32-C3 boards on AliExpress and found some marked "LuatOS-Core", so I tried to find out more about those boards and stumbled upon https://wiki.luatos.org/ when trying to find the pinout of those boards. I did not try LuatOS at all or look at it in more detail, but I thought it could be useful for your project. There seem to be implementations for the ESP32-C3 and the ESP32-S3, source is available on gitee, I did not check the licensing.

The problem with Luatos is that it tries to provide a one-size-fits-all solution (like many other similar projects). These typically don't work. A better approach is to write the low level stuff in C and then use scripting for a implementing high-level logic only. This allows to implement a safety barrier between low level control stuff and the high level functionality.

[gnuarm]

It seems like embLua is a clear winner at this point (memory footprint, functionality, language complexity, etc), at least for embedding scripting into an already existing codebase like I need.  Am I wrong?

Isn't the memory footprint of embLua rather large?  I've seen forth implementations that fit in a very few kB of flash (like ~4 kB) and around 256 B of RAM.  One advantage is that properly implemented, forth applications use very little program storage.   

Did I misunderstand the memory footprint of embLua?

[Smokey]

It seems like embLua is a clear winner at this point (memory footprint, functionality, language complexity, etc), at least for embedding scripting into an already existing codebase like I need.  Am I wrong?

Isn't the memory footprint of embLua rather large?  I've seen forth implementations that fit in a very few kB of flash (like ~4 kB) and around 256 B of RAM.  One advantage is that properly implemented, forth applications use very little program storage.   

Did I misunderstand the memory footprint of embLua?

12K RAM, 120K ROM is fine for an ESP32 I think.  If this was a smaller micro, I would probably need to reconsider.
Also the thing I definitely Don't Want is a scripting firmware that acts like an OS.  I already have my program which needs FreeRTOS because of the Wifi on the ESP32, so I don't want anything conflicting with that.

[gnuarm]

It seems like embLua is a clear winner at this point (memory footprint, functionality, language complexity, etc), at least for embedding scripting into an already existing codebase like I need.  Am I wrong?

Isn't the memory footprint of embLua rather large?  I've seen forth implementations that fit in a very few kB of flash (like ~4 kB) and around 256 B of RAM.  One advantage is that properly implemented, forth applications use very little program storage.   

Did I misunderstand the memory footprint of embLua?

12K RAM, 120K ROM is fine for an ESP32 I think.  If this was a smaller micro, I would probably need to reconsider.
Also the thing I definitely Don't Want is a scripting firmware that acts like an OS.  I already have my program which needs FreeRTOS because of the Wifi on the ESP32, so I don't want anything conflicting with that.

120 kB of flash is not what I call a small footprint.  I don't know what you mean by "acts like an OS".  Forth only does what you tell it to do.  It is very much like a scripting language, with either compiling code, or executing from a command line.  While there are forths which run on bare metal, these are less common on larger CPUs, and typically are programs under an OS.  However, forth can be the only code on a small MCU, which is why it can fit in only 4 kB of flash (what you call ROM).  There aren't many MCUs with ROM these days.

[westfw]

I think the winner is probably some flavor of BASIC, but no one wants to admit it.  :-)

It doesn't help that there is so little cross-platform standardization of low-level primitives for BASICs.

[brucehoult]

I think the winner is probably some flavor of BASIC, but no one wants to admit it.  :-)

I really don't think so :-)

The only advantage simple BASIC has is the line numbers facilitate easy program editing when you don't have a real full-screen text editor. But the line numbers quickly become an impediment to program growth. And only global variables and IF/GOTO .. just ugh.

If you add control structures and functions with arguments and local variables and structs or dictionaries then you lose the few advantages BASIC started with and might as well use Lua or a TinyC or Python or Scheme etc.

[nctnico]

It seems like embLua is a clear winner at this point (memory footprint, functionality, language complexity, etc), at least for embedding scripting into an already existing codebase like I need.  Am I wrong?

Isn't the memory footprint of embLua rather large?  I've seen forth implementations that fit in a very few kB of flash (like ~4 kB) and around 256 B of RAM.  One advantage is that properly implemented, forth applications use very little program storage.   

Did I misunderstand the memory footprint of embLua?

12K RAM, 120K ROM is fine for an ESP32 I think.  If this was a smaller micro, I would probably need to reconsider.
Also the thing I definitely Don't Want is a scripting firmware that acts like an OS.  I already have my program which needs FreeRTOS because of the Wifi on the ESP32, so I don't want anything conflicting with that.

120 kB of flash is not what I call a small footprint.  I don't know what you mean by "acts like an OS".  Forth only does what you tell it to do.  It is very much like a scripting language, with either compiling code, or executing from a command line.  While there are forths which run on bare metal, these are less common on larger CPUs, and typically are programs under an OS.  However, forth can be the only code on a small MCU, which is why it can fit in only 4 kB of flash (what you call ROM).  There aren't many MCUs with ROM these days.

That is not a very fair comparison. The 90k to 120k for Lua includes: compiler to create bytecode, math, string, table support, bit-operations and various other features like key-value pairs. For math support, the floating point library is also linked in (which is included in the size). All in all you get a fairly complete package from embLua. Micropython is even better featured with things like filesystem support and modules to deal with JSON (for example).

'Act like an OS' means that the script language is the only way to get something done which is typically not what you want because it would require the script language libraries to also abstract ALL the peripherals on a microcontroller. The latter is where these attempts / projects typically go wrong as the number of platforms they support is just 2 or 3 out of thousands of different microcontrollers out there.