Command languaje for tiny microcontrollers

[brucehoult]

167 PPI. That's the same as my 17" MacBookPro (pre "retina") on which I run 9 pt Monaco / ProFont / Ubuntu Mono no problems.

So you are telling me that a pixel drawn 9/167x25.4 = 1.37 mm height (and that's for the tallest letters, the x height is about half!)  font is legible in an useful way.

I don't have one, so I don't know how good the screen is.

AFAIK, the 17" Macbook Pro has a 1920x1200 resolution, that puts it more in the 133 ppi range;

Ah dammit you're right. And the 4k (3840x1920) 32" monitors I use on my desktop machines are 138 PPI. I was going from memory. Now I'm wondering what it is that I've used that was ~160 PPI.

in any case, a correctly rendered 9 points font will be about 3.2 mm independent of screen resolution.

This is true, but that is a terminology argument that was lost 35 years ago. The font is called Monaco 9 and is 9 pixels high no matter what the screen resolution. It is 9 pt high on the original Mac's rather low resolution 72 DPI screen.

A 9 pixels font will instead be 1.72 mm, still awfully small, but 25% larger, and overall better especially considering the difference in quality of the panels.

At 133 or 138 PPI 9 pixel high characters are fine for things that get referred to occasionally, for example "top", especially on the MBPs very good screen, though it's ok on the cheap Philips 32" 4K. For things I'm spending all day looking at (my source code) I bump it up to 11 on the MBP and 12 on the 4K.

To this I there's only one reasonable answer: I'm green with envy for your eyesight.

Uh, I dunno. I don't think it's anything special. I'll be 60 later this year and have had astigmatism since childhood, though it wasn't diagnosed until I was 35 and my eye muscles started to get fatigued after a few hours of distorting my eyeball to correct it, so I got glasses and all was fine again.

When I was a kid I only knew people would talk about details of far-away objects and I'd struggle to even see the objects.

I renewed my truck license a couple of years ago, and I'm still legally ok to drive heavy vehicles without glasses. Or fly a private aircraft.

Last time I got new glasses they did a fresh prescription, ordered the lenses/glasses, and when I got them I immediately said "they're wrong -- they're worse than my old glasses". They measured the lenses and they matched the prescription as measured a couple of weeks earlier. They told me to take them home and use them for a few days to get used to them. I did so, and came back. I said "no ... I can't use these ... the left lens needs to be rotated to the left 5 degrees and the right lens needs to be rotated to the right 10 degrees". Which I determined by holding the glasses in front of me and rotating them.

They said "We only measure the prescription to 15 degrees!". "Well, I'm not going to use these ones. I can tell the 10 year old glasses are not quite correct now, but they're better than these". They re-ordered with my corrections and the glasses came back great.

sigh. And that was a decade ago, so I'm past due to go through that again.


Having now got the correct PPI for my computer screen I'm prepared to concede that a 9 pixel font is probably too small for even casual use on a 2.4" 320x240 screen :-)

[Nominal Animal]

The font is called Monaco 9 and is 9 pixels high no matter what the screen resolution.

Well, you could have said that, then: 9px Monaco 9, instead of 9pt Monaco!  Monaco itself is a TrueType font nowadays, with the bitmap font (Monaco 9) a historic specialty in macOS.

Having now got the correct PPI for my computer screen I'm prepared to concede that a 9 pixel font is probably too small for even casual use on a 2.4" 320x240 screen :-)

Righty-o.  It's sufficiently readable for rare debugging and such, but not for casual use – especially if it is a TN panel with limited viewing angles.

(One of my forever projects is adding an 2.8" 240×320 IPS display to a MikroTik RBM33G with a 4G/LTE modem/edge firewall for rural use, to display connection status in an user-friendly manner (when a button/capacitive sensor is triggered), but also to display the IP address or error message when needed for remote debugging.  I have a very similar problem: the display is too small to display lots of stuff, so I'm going for user-friendly animated graphics instead.  But to not look utterly crappy, that needs a fast display update rate, which is borderline with the typical SPI connections to a ILI9341 controller, so I'm moving to parallel 8080-style bus comms.  Which has its own interesting problems, not the least of which is the parallel bus pinouts from various MCUs I have.  Forever project...  )

[nctnico]

I want a scripting language to make programs that test commands made with C code.

I program low level routines in C and test them until they work.
Then I want to make a program that calls this low level routines as needed, in which I will make many changes to tune it and that I do not care that it is slow in execution, but flexible and fast when programming it.
In the end I can convert that code to C once it is well debugged.

After reading the thread I think what you need is a command line interface that can execute commands that are send over some kind of communication interface (serial port is the easiest). Not a scripting language. A scripting language on a microcontroller is only useful to make easy (in field) changes to the high level functionality of a device.

Edit: a command line interface is easy to adapt to a simple form of scripting. The u-boot bootloader is an example.

[Picuino]

Yes, but a simple command line interface is not enough. I want it to be programmable as well. What I like the most so far is the Tethered Forth. It serves as a command line and programming language all in one, with very little weight inside the microcontroller. I'm going to try to develop something similar for Arduino.

[nctnico]

Take bash as another example for scripting a command line. Using an archaic language is not going to be very popular amongst people that need to work on your project

[SiliconWizard]

Is the goal to actually implement projects with this, or just to offer a new "scripting" language for MCUs (which the added "something similar for Arduino" kinda suggested, but maybe I'm reading a bit too much into it.) Just curious.

[Picuino]

The objective is to control the microcontroller from the PC, both in command line and with automatic macros.
Forth does not have to be the language of choice. In fact I am thinking of doing the same, but with Python as the PC side language. It would be a Tethered-Python.
On the microcontroller side, an interrupt will capture the commands sent by the PC and execute the function corresponding to the command sent.
The microcontroller side program I want to program it in C (not C++) so that it is portable from the Arduino to other microcontrollers like the PICs that I also use.

[Sal Ammoniac]

When I had a similar need a few years ago, I implemented a small Forth in ARM assembly language. The core was ~1000 lines and the rest was Forth source code.

The whole thing works fine--the only problem with it is that I hate Forth!

[SiliconWizard]

Forth is probably hard to beat in terms of code footprint. Micropython and Circuitpython are becoming popular, although they will certainly require more memory than Forth.

[ogden]

Unless you are fighting to shave every cent off the final product price which you dream to sell in >=5 figure quantities, code footprint of interpreter shall not be concern. Hint: cost of STM32F411-based (512KB flash) "black pill" board is around 10$.

[Picuino]

I usually use 8-bit microcontrollers because they still use 5 Volt outputs. Some 16-bit PIC microcontrollers also use 5 Volts, but they tend to have little Flash memory as well.

It is true that more and more peripherals and circuits are running on 3.3V, but there are still many 5V ones.

[DiTBho]

1024x768 is more than enough
For me

[ogden]

I usually use 8-bit microcontrollers because they still use 5 Volt outputs. Some 16-bit PIC microcontrollers also use 5 Volts, but they tend to have little Flash memory as well.

It is true that more and more peripherals and circuits are running on 3.3V, but there are still many 5V ones.

You rarely get it all. 5V I/O, tiny MCU with loads of memory and performance for decent interpreter does not exist. I would suggest to reconsider 5V I/O (of last century) requirement, think of level shifters if needed but pick modern materials, tools and solutions.

[SiliconWizard]

Yeah. Many 5V stuff? Sure. Uh... Not really. But as always, if your main tool is a hammer, all you'll see everywhere will be nails.

[Nominal Animal]

Is there an intended difference between "command languages" and "programming languages"?  Or are we really talking about query-response interfaces?
What are the practical use cases here?

For example:

Let's say that we have an input lexer-parser that converts incoming text into tokens stored in a list (or first-in, first-out stack).  Each token is either a known name, or a numerical value, and uses the exact same amount of memory (say, 32 bits).  Whenever the input is complete (for example, a newline is received), a handler function is called to handle the token list, perhaps selecting the exact function based on the initial token.

This is not a "language" per se, but a query-response interface.  If you have a perfect hash function (as I discussed earlier), the input parser does not need to keep a copy of the input string at all.  It can be a relatively simple state machine that consumes the input, and poops out a list of tokens.  The RAM use is minimal.

The property=value and property? interface I described earlier, parses initially the property name, with the following character choosing which function will eventually be called. If the character is =, then the following numerical value is parsed.  Otherwise, there should be nothing on the line except whitespace (or comments) before the newline.  This needs only a few bytes of RAM to implement, with almost everything in ROM/Flash, which is very useful for memory-constrained tiny microcontrollers.

The more shell-like form (space-delimited words as tokens), as discussed briefly before, with e.g. set property value and get property, is not much more complicated, but does need RAM for the token list.  Its state machine can actually be even simpler.  It is a lot more versatile while still having a very intuitive action/command interface as well (command [ arguments...).

Variables are easiest to implement if they always start with a sigil, say $, used also when assigned to (unlike in Bourne or POSIX shells).  Then, each token is either a known name, a numerical value, or references a variable; with variable values stored in a dictionary –– basically a set of token-value or token-token pairs.  When the hardware supports 32-bit floating-point (IEEE 754 Binary32, aka "float" in most situations nowadays), one can use NaNs (over four million distinct values) for non-numeric tokens.  This is very useful, because that way all tokens are exactly 32 bits.

In all cases, for known names, one can use a hash table in Flash/ROM for very quick lookup and no RAM used.

However, with this approach, there is no code listing, just individual commands and assignments and queries, that are acted upon when completely received, and then discarded.  You can implement short loops and repeats (as long as they fit in a single command, that is), and even conditionals and aliases, but procedures or functions would require separating the execution from parsing the tokens: you'd need to store the list of token lists, and execute them separately.  (A simple option is to use a separator token, like ; in POSIX and Bourne shells, but the big difference is separating the execution from the parser.  Technically, it is not difficult, but depending on the implementation, can be quite "tricky".  It is better to design the system for the intended model from the ground up.)

So, to get the discussion forward, what exactly do we need from the interface we're talking about?

I do not see myself needing any kind of full language with function support in a tiny microcontroller at all.  However, the command/query-response interfaces I describe here are useful, and definitely nothing new.  And they can be made with very frugal resource use, too.  Minimal code is needed to access and control any (numerical) variable in RAM.  Simple wrapper functions are needed to make commands that call firmware functions.  My uses vary from experimentation to debugging.  Granted, I do hobby stuff and a lot of personal experiments, not commercial products, so perhaps my own experience is flawed. 

(I also like full duplex more than a strict query-response interface, so that even the microcontroller can output similar format statements at any point in time (useful for timed stuff).  That is, responses not necessarily being in the order of the queries, but in the order of completion, with additional events possibly reported in between.  This means that I also like the way e.g. Gcode lets one specify an identifier for each command or query, with that identifier included in the response.

Looking at "languages" like G-code and HPGL is useful, because they were (and still are) parsed and processed by 8-bit microcontrollers.  Some of their features are historical baggage, but some are a careful balance between human readability and ease of machine processing.

[Picuino]

At first I was looking for a language similar to BASIC or FORTH so that the microcontroller could interpret it and give me control over its processes.
Now I have realized that a language interpreted by the microcontroller takes up too much space, must be too simple and makes no sense.
Since the interpreters on the microprocessor are hardly going to execute instructions faster than data coming in over a UART or SPI channel, it is better to move the programming language to the PC and send codes to the microcontroller that will be quickly executed.
It is not even necessary to implement loops or conditionals. That can still be done by the PC. The only thing really necessary is to implement instructions in the microcontroller and a dispatcher that, depending on the arrival of one code or another, calls the different functions.

For example the PC sends the code 0x20 and the microcontroller reads the analog input 1 and returns the value readed.
The PC sends code 0x40 and the microcontroller sets digital output 1 to high level.
Etc, etc.

What I now need to know is how to optimize and standardize the codes sent to the microcontroller so that I don't have to rewrite the PC and microcontroller programs every time I make a new application.

[tggzzz]

You are using the microcontroller to control something ultimately using reads and write operations, with some additional processing to determine the specific reads and writes.
What is the maximum allowable latency between one such control action and the next?
What is the maximum latency for comms to and from the microcontroller and a PC application?

Have you considered that a very fast edit-compile-download cycle would be as effective debugging environment as an interpreter?
... especially if there is an additional REPL loop in the microcontroller for human-speed exploration of what's happening?

That would enable you to:

• have compact and low latency simple read/writes/processing in the form of compiled functions running in the MCU
• explore the next "higher" level of composite actions by typing at the keyboard
• when that next level is sufficiently understood (i.e.  debugged), compiling that and downloadind it

Rinse and repeat as you increase the complexity/sophistication of your actions.

I have no specific environment in mind, but I would be surprised if you couldn't find a Forth that could do it.

[DiTBho]

Micropython and Circuitpython are becoming popular, although they will certainly require more memory than Forth.

Still problematic on my Casio SH3-hacked calculator.
Too fat!!!

[IDEngineer]

An interpreted language allows us to program and test with the microcontroller faster than with compiled languages, but there are not many options to choose from. Here I present a few and the solution proposed by zserge.

I know I'm late to this thread, but OMG... we went through this exercise back in the 80's. "Compiled languages are too much, let's use an interpreter." "It will be faster, smaller, easier to learn, etc." We wasted man-months on this. In the end, it was just easier to use C for sooooo many reasons. It's an, um, "interesting" exercise and it's your time to spend, but having gone down this road I would never ever recommend it to anyone.

[nctnico]

But back in the 80's Lua and Python did not exist and cpu power was limited. Nowadays a $1 ARM microcontroller runs circles around any PC from the 80's. On top of that scripting languages have huge ecosystems as well. IOW you can't compare like you do.

[westfw]

back in the 80's Lua and Python did not exist

We should decide what we mean by "tiny microcontrollers."

Lua and Python are pretty big by many standards (does Lua even run on smallish ARM chips?)
No one has mentioned PCode, or the pre-tokenized BASICs.

[IDEngineer]

But back in the 80's Lua and Python did not exist and cpu power was limited.

I was speaking more about the human factors, not the hardware. It turned out to be easier to coordinate the use of traditional compiled languages across a staff of Engineers.

We tried the same thing for writing test code... same result.

As with most things, the humans are usually the limiting factor.

[NorthGuy]

IMHO, interpreted languages is pure bloat with no benefits.

Many people seem to be obsessed with languages thinking that a particular choice of language makes them more productive or similar non-sense. This reminds me of people on the golf course who buy new modern drivers every year and swear they then can drive longer and play better because of their new driver, but if you look at their scores, they're still the same.

[tggzzz]

IMHO, interpreted languages is pure bloat with no benefits.

Many people seem to be obsessed with languages thinking that a particular choice of language makes them more productive or similar non-sense. This reminds me of people on the golf course who buy new modern drivers every year and swear they then can drive longer and play better because of their new driver, but if you look at their scores, they're still the same.

All languages are interpreted, but the level varies.

For example, Pascal pCode was interpreted very successfully on 8080 class machines.

A current example: if you think assembler code (and hence C/C++/etc) isn't interpreted on moderm x86-64 processors, then you don't understand the implementation of the processors. Each generation of processor has a significantly different interpreted, the details of which are not released by intel. The ability to change the interpreter is how intel fixes bugs on installed x86-64 processors.

The ability to have a fast interactive mechanism for "trying out" code is key to productivity. You cannot appreciate that until you have experienced it. That can be a conventional interpreted language or some modern compiled languages (not C++, not Fortran, not C). The lack of facilities in many languages lead to "Greenspun's Tenth Rule of Programming: any sufficiently complicated C or Fortran program contains an ad hoc informally-specified bug-ridden slow implementation of half of Common Lisp."

But I agree with you to the exent that the specific language is unimportant.

[PlainName]

An interpreted language allows us to program and test with the microcontroller faster than with compiled languages, but there are not many options to choose from. Here I present a few and the solution proposed by zserge.

I know I'm late to this thread, but OMG... we went through this exercise back in the 80's. "Compiled languages are too much, let's use an interpreter." "It will be faster, smaller, easier to learn, etc." We wasted man-months on this. In the end, it was just easier to use C for sooooo many reasons. It's an, um, "interesting" exercise and it's your time to spend, but having gone down this road I would never ever recommend it to anyone.

I am kind of following this thread because it might be of interest. As an example use, I am thinking of our garage light controller which is currently a C application that reads a config file to figure out how to control the lights. To change things you could just upload a new config file. But... sometimes I want to try something, or have something additional occur, temporarily. Maybe it's a special circumstance, or I'm just interested in how often a particular event occurs, or maybe I want to know the temperature (each node has a thermometer and the protocol for accessing them is defined but currently not used).

It would be cool to fire up the web browser and type some script into an edit box and run it. Or remove a previously entered one. Don't really want to get out the compiler and libs and everything and do an OTA upgrade just to check something, then an OTA downgrade...

What I like the most so far is the Tethered Forth.

Never used Forth (the one time I came across it, a 40x20 or similar editing screen was enough to put me off), but I could learn to live with it. However, TF is written in assembler, and I think something that might be ported should be written in C since not every destination will be an ARM-based MCH. Portable Forth is apparently ANSI C, but I don't know whether it would be small enough.

I like the idea of and onboard language, over just a bunch of functions a PC could call via a comms link, because it is standalone. Unless you intend to make an accessory for a PC, you would want to be able to tell it to do stuff and have it keep doing that.