Lightweight Scripting Language for MCUs

[gnuarm]

Bootloader? You mean it wasn't already in the core memory when power was reapplied?

No, the Space Sciences department across the street had a prototype doppler weather radar that regularly corrupted the core memory.

Those durn pesky EMC requirements!!!

[techman-001]

What is important to say is the user only writes

Code: [Select]

RCC_CR.
RCC_PLLCFGR.
RCC_CFGR.
as the command in the terminal..

Sure, that's one way to do it but I rarely type in anything at the terminal. I might type in "rcc." to get a instant upload of the registers as they are *right now* but I do almost everything in my editor.

Many people here think that one types on some old serial terminal and waits for Forth to reply via a slow UART like C users do with printf debugging.

The reality is that Forth often leads with new things that other systems don't have. Some very smart Forth people are always experimenting with new concepts and pushing the boundaries.

For instance my 'terminal' has no USART component because it works over SWD with full flow control, via USB. It's so fast that I can dump the (pretty printed, formatted) contents of registers as seen in my previous post to my screen in a few microseconds.

You can have the same thing with C if you buy an expensive Segger, but all I need is a STM32 Disco or Nucleo board for the SWD/USB dongle.

I type the Forth source code I want run (in the target MCU) in my editor and when I hit the 'make' button in the editor toolbar, the source is uploaded to the mcu faster than I can see on the SWD terminal screen. Because it's too fast to see we have had to add a terminal beep for warnings and an automatic upload abort for errors.

We want it even faster and are working on a SPI terminal atm plus a bunch of other exciting development aids, such as using an external 16MB SPI flash to store Forth development aids for use with smaller Cortex-M0 devices with only 64KB flash.

These aids might be the entire SVD2FORTH peripheral register pretty print Words, the entire Bitfield Words, ascii pictures of pinouts etc. If we wanted to leave the $2.50 SPI memory chip with the device, we could include schematics and diagnostic flowcharts for the tech servicing it a decade later.

We have already adapted to the new paradigm of MCU's with no internal flash such as the RP2040 where we now store multiple dictionaries in the external QSPI flash.  Mecrisp-Pico Forth was released on 23 Feb 2021, only a month after the RP2040 Pico itself was released for sale. This included the time it took to actually buy Pico devices for development.

In this 'golden age of embedded' where MCU's have never been cheaper, faster, or more featureful, Forth is flourishing.

[alexanderbrevig]

In case it is interesting to some of you, I've made my own small interpreted language that I use from time to time to have a flexible in system debugging and exploration.

It is a work in progress thing, for my personal use. But PRs are welcome if you find this interesting 

https://github.com/AlexanderBrevig/yalex
https://github.com/AlexanderBrevig/yalexide <- the editor/playground for prototyping scripts for larger projects.

There's an example that works on the Arduino Uno (as a proof of concept for small 8bitters) but I tend to use it on larger projects on larger  μCUs (ARM and PIC32 in my case).

[iMo]

In order to write a celestial navigation app all you would need is a mechanical teletype worst case.. Would be better if you got a PC which can do copy/paste and work with .txt files, however 

What do you have against paper tape?  ..

Linux and teletype.. https://hackaday.com/tag/teletype/

[techman-001]

In order to write a celestial navigation app all you would need is a mechanical teletype worst case.. Would be better if you got a PC which can do copy/paste and work with .txt files, however 

What do you have against paper tape? 

Other than the High Speed Paper Tape reader sometimes shooting (a box full of) paper tape across the room like a massive streamer, and having to clean it up, check for tears in the tape etc ?

[SiliconWizard]

In case it is interesting to some of you, I've made my own small interpreted language that I use from time to time to have a flexible in system debugging and exploration.

It is a work in progress thing, for my personal use. But PRs are welcome if you find this interesting 

https://github.com/AlexanderBrevig/yalex
https://github.com/AlexanderBrevig/yalexide <- the editor/playground for prototyping scripts for larger projects.

There's an example that works on the Arduino Uno (as a proof of concept for small 8bitters) but I tend to use it on larger projects on larger  μCUs (ARM and PIC32 in my case).

At first sight, it kind of looks like a cross between Lisp and Forth.

[abyrvalg]

Recently I've done something similar to OP's question (scripted/interactive fiddling with STM32 peripherals) without running any MCU code at all, accessing memory-mapped registers over SWD instead.
A simple Python wrapper over pylink (Segger J-Link Python lib) halts the CPU at start and provides low level functions like read32(addr). Higher level classes with reg name properties provide things like stm.GPIOE.BSRR = val. All that with a comfort of a full PC Python (IDE, highlighting, autocomplete etc. At the end I even wrote a simple hw tester GUI polling & displaying ADC values as bars and toggling GPIOs with buttons).
Actually, this approach could be combined with some init code running on the MCU (i.e. use HAL to do complex init, then enter an endless loop and let the PC side continue the job).

[elecdonia]

Bootloader? You mean it wasn't already in the core memory when power was reapplied?

In the late 1970’s I purchased a heavily used and abused Data General NOVA system where the core memory had burned up (literally). The previous owner replaced it with (then new-fangled) dynamic RAM. No data retained during power off. I manually entered the NOVA bootloader thousands of times from the dreaded row of 16 toggle switches. Fortunately the entire bootloader was something like 28 words in length. And yes, its program input format was paper tape (8 level).

[coppice]

Bootloader? You mean it wasn't already in the core memory when power was reapplied?

In the late 1970’s I purchased a heavily used and abused Data General NOVA system where the core memory had burned up (literally). The previous owner replaced it with (then new-fangled) dynamic RAM. No data retained during power off. I manually entered the NOVA bootloader thousands of times from the dreaded row of 16 toggle switches. Fortunately the entire bootloader was something like 28 words in length. And yes, its program input format was paper tape (8 level).

I entered the bootloader on a Nova with core memory quite a few times. Bugs meant it corrupted with annoying frequency, and when it had changed the whole thing was usually scrambled. I used other mini-computers where I had to re-enter the boodloader from time to time, but in most case only a few locations needed correcting. With the Nova and a Perkin Elmer machine it always seemed to need the whole thing re-entered. Its quite a tedious thing to do, but if you do it often enough you get reasonably quick.

[JoeyG]

+1 mmbasic

[brucehoult]

In case it is interesting to some of you, I've made my own small interpreted language that I use from time to time to have a flexible in system debugging and exploration.

It is a work in progress thing, for my personal use. But PRs are welcome if you find this interesting 

https://github.com/AlexanderBrevig/yalex
https://github.com/AlexanderBrevig/yalexide <- the editor/playground for prototyping scripts for larger projects.

There's an example that works on the Arduino Uno (as a proof of concept for small 8bitters) but I tend to use it on larger projects on larger  μCUs (ARM and PIC32 in my case).

At first sight, it kind of looks like a cross between Lisp and Forth.

Close to PostScript and Logo ("forward Polish") too.

Could use anonymous blocks for if/else and loops

[member_xyz]

https://cicciocb.com/annex32help/V1.482/

Basic with lot of built in functions and drivers

[SpacedCowboy]

I had a large hardware project (that you now all know about, but has yet to be officially released). Back in the prototype days, it existed on a bunch of largest-on-the-planet FPGAs all wired up with optic interconnect, and we had a Zynq as the boot-controller.

I wrote the firmware for all this, and chose Lua as the scripting language - basically wrote an OS around Lua rather than unix's /bin/sh. I implemented a virtual filesystem interface (flash for permanent storage, Ramdisk mounted as /tmp), so we could store data between runs, and it even ran a boot sequence (looked in /etc/rc.d for scripts prefixed by a number and ran them in numerical order). That handled bringup because all the things we needed to control were bound to objects in the Lua domain, so could be scripted. Lua provided the control structures that let you write useful scripts ("read the value of this register on that i2c device on the specified bus, is it X, then do Y else do Z").

Using the system interactively, there was a Lua shell interface that talked over thunderbolt (via a custom PCIe kernel extension) to a connected Mac, and since all the hardware was bound to objects in the Lua shell people using it didn't have to get me to redo some firmware when they wanted to fiddle with something, they just used the shell to either change things interactively, or rewrite the file in /etc/rc.d and reboot the device if it was a "boot up" stage thing they wanted to change.

I had some fun with this project, even implemented stdio redirection, so you could run a command on the device, and have the output appear on the Mac - or vice versa - which turned out to be really useful for long runs of data collection and interactive monitoring (drawing a graph on a monitor on the wall) over time.

Lua was pretty much perfect for this. Expressive enough to make it a useful language to have around, and completely trivial to bind hardware to objects so the hardware could be easily interacted with. 10/10 would (and did) use again.

[ag123]

not really scripting, but do anyone mess with Rust
https://blog.mbedded.ninja/programming/languages/rust/running-rust-on-microcontrollers/

and found it any 'better' than C/C++?

[audiotubes]

There's nothing wrong with stack models--I was an early HP calculator user (HP35) and like RPN. In fact, I've only ever used HP calculators. An HP calculator only has a four level stack, which makes it rather easy to remember what's on the stack. With Forth programs more complex than trivial one liners you have more to keep track of, and this is compounded by operators that manipulate the stack in ways you can't do on an HP calculator (like 2over and 2rot).

The RPN models have a 4 level stack. The RPL models (HP 48/49/50, maybe others) have an infinite stack and all the stack manipulation words anybody could ever want.

[gnuarm]

There's nothing wrong with stack models--I was an early HP calculator user (HP35) and like RPN. In fact, I've only ever used HP calculators. An HP calculator only has a four level stack, which makes it rather easy to remember what's on the stack. With Forth programs more complex than trivial one liners you have more to keep track of, and this is compounded by operators that manipulate the stack in ways you can't do on an HP calculator (like 2over and 2rot).

The RPN models have a 4 level stack. The RPL models (HP 48/49/50, maybe others) have an infinite stack and all the stack manipulation words anybody could ever want.

Oh, like Forth?  Cool. 

[thermistor-guy]

I often need to do some quick prototyping and testing of sensors on some of my MCU platforms and I find myself writing the code in C and compiling it, downloading it, etc. I'd like something like a small scripting language I can interact with over a serial link to a board to send commands to a sensor, look at the output, etc.
...

For controlling my test instruments and acquiring data, I run GNU awk scripts on an rpi3. Some of these tests can take a week, so it's useful to have
an rpi dedicated to that purpose.

Gawk is useful too for processing the raw data.

Anyone know of a scripting language small enough to run on an ARM Cortex-M4 and not require too many resources?

In my setup, the approach I would take is to convert the target board (with sensors) into a test instrument, by giving it
a minimal SCPI-like command set. Then control the target board from an rpi that runs the scripts (Gawk, Perl, Forth, ...).

In cases involving  large amounts of data, the rpi saves it to a NAS with redundant drives.

[brucehoult]

There's nothing wrong with stack models--I was an early HP calculator user (HP35) and like RPN. In fact, I've only ever used HP calculators. An HP calculator only has a four level stack, which makes it rather easy to remember what's on the stack. With Forth programs more complex than trivial one liners you have more to keep track of, and this is compounded by operators that manipulate the stack in ways you can't do on an HP calculator (like 2over and 2rot).

The RPN models have a 4 level stack. The RPL models (HP 48/49/50, maybe others) have an infinite stack and all the stack manipulation words anybody could ever want.

My earlier HP28s has stack limited only by the 32 KB of RAM.

The HP-41C from 1979 (and CV and finally CX in 1983) were the last flagship HP calculators to have a 4-level stack, though it was alphanumeric and could hold (short) strings.


Note that the INMOS Transputer microprocessor had a three level stack.

[audiotubes]

You're right that the later RPN models were not flagships but they had no shortage of features, just that the build quality started to go downhill. But the 41 also did not have the build quality of the HP 67 and the models which came before the HP 41, although the 41 the first HP calc with and LCD display (which by the way, we did not trust at the time!)

The HP 15C is RPN, has flagship status (at least in recent years) and has had 2 special editions come out, the last one this year.

The 35s introduced in 2007 has a 4 level stack and there are other "new" models that also have RPN (4 level stack) at least as an option (30b, etc.) There are some RPN models still in production.

Rather than dates and flagship/not flagship, it's useful to distinguish between RPN and RPL variants. The RPN variants have a 4 level stack, RPL variants have an infinite stack.

Anyway, here's a piece of code I wrote to convert roman numerals to real numbers (useful for dates on those Goldwyn Meyer movies). Yes, RPL is very similar to FORTH but also with some interesting differences.

One major problem is that there isn't any way to keep comments in the code and you probably wouldn't be able to stand keying them in anyway on a calculator keyboard. There are 3rd party tools, but the standard setup is that everything is device-hosted. They have built in kermit servers so I usually try to kermit a copy of my code to a PC and then annotate the file. Not a great workflow, but good enough for smallish projects.

Code: [Select]

@
@ R->A2 (second version)
@ Converts roman numerals to real value
@ Roman numeral input can be single or double-quoted
@ ( roman numeral string --> real value )
@
%%HP: T(3)A(R)F(.);
\<< "" + RCLF SWAP { }
SWAP 1 CF
  WHILE DUP SIZE 0. > REPEAT
     DUP HEAD
     CASE
        DUP "M" == THEN 1000. END
        DUP "D" == THEN  500. END
        DUP "C" == THEN  100. END
        DUP "L" == THEN   50. END
        DUP "X" == THEN   10. END
        DUP "V" == THEN    5. END
        DUP "I" == THEN    1. END
        # 202h DOERR
     END
     SWAP DROP ROT SWAP +
     IF 1 FS? THEN
        DUP SIZE 1 - GETI ROT ROT GETI SWAP DROP ROT DUP2
        IF > THEN
           NEG SWAP DROP SWAP DUP SIZE 1 - ROT PUT
           1 CF
        ELSE
           DROP2
        END
     ELSE
        1 SF
    END
    SWAP TAIL
  END
  DROP DUP SIZE 1
  IF == THEN
     1 GET
  ELSE
     \GSLIST
  END
  SWAP STOF
\>>


[DiTBho]

I had a large hardware project (that you now all know about, but has yet to be officially released). Back in the prototype days, it existed on a bunch of largest-on-the-planet FPGAs all wired up with optic interconnect, and we had a Zynq as the boot-controller.

I wrote the firmware for all this, and chose Lua as the scripting language - basically wrote an OS around Lua rather than unix's /bin/sh. I implemented a virtual filesystem interface (flash for permanent storage, Ramdisk mounted as /tmp), so we could store data between runs, and it even ran a boot sequence (looked in /etc/rc.d for scripts prefixed by a number and ran them in numerical order). That handled bringup because all the things we needed to control were bound to objects in the Lua domain, so could be scripted. Lua provided the control structures that let you write useful scripts ("read the value of this register on that i2c device on the specified bus, is it X, then do Y else do Z").

Using the system interactively, there was a Lua shell interface that talked over thunderbolt (via a custom PCIe kernel extension) to a connected Mac, and since all the hardware was bound to objects in the Lua shell people using it didn't have to get me to redo some firmware when they wanted to fiddle with something, they just used the shell to either change things interactively, or rewrite the file in /etc/rc.d and reboot the device if it was a "boot up" stage thing they wanted to change.

I had some fun with this project, even implemented stdio redirection, so you could run a command on the device, and have the output appear on the Mac - or vice versa - which turned out to be really useful for long runs of data collection and interactive monitoring (drawing a graph on a monitor on the wall) over time.

Lua was pretty much perfect for this. Expressive enough to make it a useful language to have around, and completely trivial to bind hardware to objects so the hardware could be easily interacted with. 10/10 would (and did) use again.

this post should belong to the thread where I asked "with what?" can I replace Bash scripting

[DiTBho]

yalex

don't take personal, but this stuff is exactly what I do find good motivation when you are a student and need to open your mind by experimenting with different things, and awful when you become a pro and need to develop, debug and maintain that stuff, even because usually no one in your co-workers is willing to get their hands on it.

[audiotubes]

this post should belong to the thread where I asked "with what?" can I replace Bash scripting

I use SNOBOL4 for that, but I realize that may raise some eyebrows

[brucehoult]

this post should belong to the thread where I asked "with what?" can I replace Bash scripting

I use SNOBOL4 for that, but I realize that may raise some eyebrows

Markov algorithms ftw.

[SpacedCowboy]

this post should belong to the thread where I asked "with what?" can I replace Bash scripting

You're right

I just read that thread (hadn't seen it before) and for a custom-everything OS, Lua was pretty much perfect for the startup sequence, and came with the added bonus of making all the interactive use a snap, because I'd already done all the work of binding the I2C, SPI, DMA etc to objects within the Lua environment. Writing a shell around this was trivial.

I can't say I'd forgo bash on a real system, but to do it on a bare-bones minimal system, it really worked well.

[nctnico]

So far I have used both Micropython and Lua to implement high-level logic in embedded software.

The main problem I found with Micropython is that it is an all-or-nothing approach. With a bit of hacking I have succesfully integrated Micropython into a C project that uses FreeRtos but the fact Micropython uses Python scripts to pre-process strings make the whole build setup less than perfect (calling external, modified Micropython make files to make it all work). Also creating my own Python modules isn't as straightforward as I would like.

I'm also moving ahead with Lua for use on microcontrollers. I'm working on a fork that can be used in an OS-less round robin style scheduler. The big advantage is that I can basically copy all the Lua stuff into a C project, set the include directories and it will compile just fine. I second SpacedCowboy's opinion that Lua is easy to integrate into C/C++ programs to add scripting functionality.