Creating a portable and small I/O lib.

[Doctorandus_P]

I am having a lot of difficulty in deciding what sort of I/O abstraction I want to use in my own projects.
something like the "digialwrite()" in arduino, but then something that works properly and with zero overhead (compared to machine code)

When I started uC programming with the Atmel AVR's (10+ years before arduino existed). I used simple direct port manipulation:

Code: [Select]

#define LED_PIN (1<<4)
#define LED_PORT  PORTD

// Turn the LED on:
LED_PORT &=~ LED_PIN;
It works, is simple and fairly portable among different uC families, but it does not look pretty in the software, and I am looking for a method that is more readable in the source code. A few days ago I did a little experiment with a C++ class for an STM32.

Code: [Select]

Tgpio::Tgpio( GPIO_TypeDef* portname, uint32_t bitname) {
the_port = portname;
the_bit = 1 << bitname;
}

void Tgpio::set( void) {
the_port->BSRR = the_bit;
}

void Tgpio::reset( void) {
the_port->BSRR = the_bit << 16;
}


static Tgpio led( GPIOC, 13);
led.set();


And of couse the class declaration in the header file:

Code: [Select]

class Tgpio {
public:
Tgpio(  GPIO_TypeDef * portname, uint32_t bitname); // Constructor.
void set( void);
void reset( void);

private:
GPIO_TypeDef * the_port;
uint32_t the_bit;
};

This works, you can easily define more functions in the class, it looks proper in the code, and with the code completion, you can simply type the name of the led, a dot and your IDE gives you options what you can do with the led. A disadvantage is that it has runtime code overhead. I know something similar can be done with templates, but template syntax is quite difficult to write and debug, and I'm not sure if code completion works (in a bunch of different IDE's).

Now I am wondering how much of the code can be transferred from the runtime to compile time by using constructs such as const constexpr, declaring the class as static etc.

[ejeffrey]

As long as you compile with optimizations simple functions like this will be inlined 100% as long as the function bodies are visible.

You need to put them in the header file and either explicitly declare them as online or include the body in the class declaration which implicitly declares them inline.

[wek]

> it does not look pretty in the software

Who cares?

JW

[NorthGuy]

Every programmer has OCD. Some more, some less. This is Ok.

One useful thing to realze is that: Your OCD is not your friend, your OCD is an enemy.

I use macros. Like

#define BLUE_LED_ON something-here

it's very easy to collect all the definitions into a special file, and simply edit it when you move to a different board, or different platform. This gives me endless opportunities - for example, if a new board doesn't have a blue LED I can generate nothing.

[nctnico]

I agree with the macros Northguy is suggesting. I'm using these on any platform I'm writing embedded software for. I have 1 file (typically called hardware.h) where all the GPIOs and hardware related defines go into. GPIOs are very project specific so portability is not really an issue here.

#define PIN_BUZZER_EN (1<<10)
#define SET_BUZZER_EN()            GPIOC->BSRR = PIN_BUZZER_EN;
#define CLR_BUZZER_EN()            GPIOC->BRR = PIN_BUZZER_EN;

And use the set and clr macros throughout the rest of the core

[hans]

I think one of the pros of having a GPIO object is you can incorporate it into device drivers as well. For example, you may have a Wiznet W5500 ethernet controller, that has a SPI port that needs some transactions (framed by CS) and has an interrupt request input. If you can pass a Tgpio reference to the GPIO used CS+nIRQ, then you actually use that same source code for various board or project builds. If its an object and nicely instantiates, you could even instantiate multiple W5500s quite easily and add project-specific packet handling, etc.

The Tgpio object would need to also expose an interrupt-on-change structure, which may need some manual work though for the IRQ handler (it may be shared with other GPIOs). I'm still looking for a neat way to address that from C++.
Secondly, you would need to decide on when to access the SPI device, as doing SPI accesses from IRQs introduces some architectural limitations (guaranteed bus access). Alternatively, you could issue a RTOS event to some task that then executes the IRQ request with less architectural limitations (e.g. adding a semaphore on SPI driver objects), but potentially latency issues due to priority inversion (application specific though).

Another application could be assigning an I2C bus to random pins. Some MCUs can pin-mux a driver to those pins, then use the hardware driver. If not, you could have the firmware fallback automagically to a soft driver. But its questionable if those kind of implicit decisions are desirable at all. From an Arduino "lets build fast" perspective, I think being able to do that compile-time (and also reserve e.g. I2C3 for a certain instantiation) would be an interesting challenge to build.

Certainly for LEDs and buttons its much easier to create a small header file instead. You may only add 2 buttons A-B and 4 LEDs A-D on a PCB, so its easy to always label them like that and use those labels in software. But its not the only use-case for GPIOs and I can certainly see why a low overhead implementation can be useful.

[cv007]

A disadvantage is that it has runtime code overhead

For a 32bit mcu, it really doesn't matter. So you took a few extra cycles to toggle your led, and a few extra bytes of ram/flash. Make code that is readable and reliable, let the compiler handle the details. Nothing wrong with pursuing optimal code, its just not a great use of time when in most cases you really gain nothing.

Start at the usage side, so you can write code the way you would like it to 'look and feel', then work out the details to get there.

I have c++ code for avr, stm32, nrf52, efr32, ra, etc., and it all looks basically the same. There are hardware differences so there will be differences, but for the most part the usage can be similar for each mcu.

sample GpioPin class for stm32, G031 in this case (no pin irq stuff in this version, although is not much more code)-
https://github.com/cv007/NUCLEO32_G031K8_chrono/blob/main/include/GpioPin.hpp
This repository was meant to test the chrono library, among other things like ownership and atomic types. Also note that the GpioPin class in this version uses (my) Atomic types so if pin properties happen to be manipulated in any irq code the operation will be atomic (pins share registers such as AFR, MODE, etc). I doubt anyone does this, and there may be little need to do, but it is the correct thing to do unless you warn the 'user' not to manipulate pin properties in irq's.

You can use 'normal' classes and get the compiler to produce optimal code by keeping the object inside a local scope, but again its not worth the trouble and you end up with dual usage plus an additional decision each time (do I want fast or normal). What is worth the trouble, is to use const/constexpr/static whenever you can just like any other C/C++ code. Templates are useful, but if used to get always optimal code produced such as with a gpio class, you will end up with templates everywhere as each pin will be a specific type (want to pass a pin to a function? function will need to be a function template to handle any pin). Eventually you get better at seeing when to use/not use templates, and will tend to avoid if possible.

When I was learning C++ on the pic32mm, I knew a lot less than I know now but that code was reliable and everything I tried worked usually the first time. I wrote a driver for every peripheral including usb (a driver for each datasheet chapter, 1hpp+1cpp file). I knew nothing about templates, could not really read mips so did not spend much time worrying about what the compiler produced (and the pci32mm was 256k). It was straight forward, easy to read and use, and in some ways probably as good as what I do now (more knowledge = more clever = usually worse, realize its getting worse, work your way back to more simple, better again). This is when I also had a goal to rid myself of defines, as they have a tendency to take over and you end up with a preprocessor 'language' as large as your mcu language in use. That complete pic32mm driver code had only a few defines in the usb header, as I could find no good way to handle string creation other than with macros. I still avoid defines if possible because I just dislike them for various reasons (and its easier to do without in C++).

You can also use the online compiler to try out ideas-
https://godbolt.org/z/768nEcGjr
I copied/pasted some code, modified to remove atomic types, etc. Also added ability to add always_inline option to GpioPin functions, to see what difference it makes, although you do then start down another path of user optimization. You can also see code produced for a local object vs a global object, and also try out an array of pins. It is a great way to test out code, except you will not have mcu headers available so have to just deal with it in some way (typically working on limited pieces of code so can add enough info to get the code to work, as in the above examples).

[tellurium]

Very similar approach to the NorthGuy's suggestion, but using "static inline ...." functions instead of macros.
Example: blinky baremetal projects for various architectures, each having almost identical main.c
The abstraction for every arch is kept in hal.h :

https://github.com/cpq/bare-metal-programming-guide/tree/main/templates/blinky

That hal.h implements the following "cross-platform" API:

gpio_{init,read,write,toggle}
uart_{init,read_ready,read_byte,write_byte,write_buf}
rng_{init,read}

[Doctorandus_P]

I appreciate your responses, and my intention is of course for more then blinking LED's.

I am also beginning to realize that I may have bitten off more than I first expected.
Yesterday I converted my HD44780 lib to be used with both the old AVR's and STM32 and it did not matter much whether I used inline functions of macro's. It was fairly straight forward as a warmup.

Today I started on porting an ISR driven uart library with RS485 transceivers and some intricate parts for collision avoidance and starting and stopping of ISR's and that proves more troublesome. I'm starting to think that maintaining separate drivers for AVR and STM32 may be the better option. Or at least use the AVR code as a very loose start for the STM32 variant, then write it independently and see later if they can be merged into one again.

At the moment I am also quite struggling with Eclipse in the stm32Cube, and though I have done a bunch of blinking led level projects with STM32 before, I'm just starting to comprehend how big the difference is between these beasts and old AVR uC's I am familiar with.

[SiliconWizard]

Whether you use macros or "inline" functions, I highly recommend taking the approach both NorthGuy and nctnico recommended here, which is to abstract at the functional level of the hardware you're dealing with, rather than at the MCU level.

What that means is that (of course there are exceptions and it's not set in stone) abstracting all MCU peripherals, including GPIOs, first, is usually a big waste of time. Vendors do that with their provided HALs because that's obviously the only level at which they can abstract (although some already do it better than others), not targetting any specific application in particular, but as a firmware developer, doing this is a waste of time unless you want to work for the MCU vendor.

So for instance, as they showed above, if you want to control some LEDs, write macros or functions that abstract the control of said LEDs, rather than writing functions that abstract GPIOs in general, with all their features, first, and then call those abstractions to drive the LEDs. You get the idea.

[westfw]

like the "digialwrite()" in arduino, but then something that works properly and with zero overhead

A problem is that it's actually pretty trivial to implement "digitalWrite()" with "zero overhead" just by forcing restrictions that the Arduino implementers wouldn't put up with.  (It's been done many times, both with and without C++style objectification.)
Once you realize that, it becomes a matter of diminishing returns.  Especially on 32bit RISC architectures.

[wek]

write macros or functions that abstract the control of said LEDs,

That to me sounds pretty much concrete.

abstracting all MCU peripherals, including GPIOs, first, is usually a big waste of time. Vendors do that with their provided HALs because that's obviously the only level at which they can abstract

It's exactly the same waste of time - except that the concept and the moniker sells these days, so that justifies that waste of time from those vendors' point of view.

That everybody loses with this approach, is none of the beancounters' concern.

JW

[Siwastaja]

A disadvantage is that it has runtime code overhead

For a 32bit mcu, it really doesn't matter. So you took a few extra cycles to toggle your led

Annoyingly, sometimes it matters. This sometimes is only like 1-5% of time, but that is enough to destroy the whole idea of elegant abstraction that can be used everywhere. If you have to bypass it even once in your project, you are duplicating and maintaining two sets of IO code, which is practically doable, but stinks from the elegance viewpoint which was the only reason to do this.

For example, this is exactly the case in my current project. I made a nice abstraction which reduces all ports and their pins into a single number which is easy to configure. In the beginning I assumed those "few cycles" are totally meaningless, but lo and behold, now I have a One-wire module and a bitbanged UART interface which both need working around that nice abstraction. Of course it's manageable, but it totally destroys that satisfaction you got designing the "nice abstraction", which, as NorthGuy correctly points out, is more OCD than actual engineering.

[wek]

I don't think it's OCD. Or, more precisely, it to certain extent undoubtedly is; but there's more behind it. I have no proof for this, but I believe this "abstraction" trend stems from academia, and simply it's a misunderstanding of what microcontrollers are, probably because of lack of first-hand experience.

Of course "abstraction"  - which in fact means "leveling the ground by defining the least common denominator" - decreases the number of details the prospective programmer has to learn (thus increases productivity for a certain class of problems), and increases chances of portability. These traits are sought after in many settings, and have proven to be beneficial in the realm of "big computers". There's also truth in that microcontrollers grew big enough to play role of "big computers" in many regards (think graphics, mass storage, "big" networking).

But when it comes to "control" as in "microcontroller", abstraction gets into way more than it helps.

2 eurocents

JW

[nctnico]

A simple rule of thumb: any software layer should add functionality. If a layer doesn't add functionality then it should be left out. If you still need a layer that doesn't add functionality, then your design is flawed.

[wek]

A simple rule of thumb: any software layer should add functionality.

How easy.

Now define functionality.

JW

[Siwastaja]

I would refine: add or modify functionality.

First of all, this rules out the classical boiler-plate layers, those we were extensively taught in Uni in early 2000's "this is what abstraction is all about": manually writing getter and setter functions which access private variables, simply making them public through indirection which adds nothing:

Code: [Select]

class bullshit
{
   private:
      float penis_length;
   public:
      float get_penis_length() {return penis_length;}
      void set_penis_length(float l) {penis_length = l;}
}

Obviously having to do this signifies that
(1) one who does this is full of shit, or
(2) the language/paradigm they use is full of shit so they are forced to do this, or
(3) both

Adding functionality is simple. This would be printf(). It adds functionality in form of built-in conversions from binary (in-memory) to ASCII text, with all sorts of formatting options, hugely useful.

Then there is modifying functionality. Now if the functionality you want is to wiggle an IO pin, then the mapping would be: wiggle command in -> wiggle command out. There would be no added functionality per se. But the exact details of a "wiggle command" could change. Example:

Code: [Select]

digitalWrite(11, 1); --> PORTB |= 1<<4;
The benefit is here the possibility to use the same interface (digitalWrite) for something which does not have PORTB but something else. We are modifying the functionality. But then one needs to be careful, some of the worst abstractions are of this type. STM32 init structures are example of total fuckup: you have to read every page of reference manual and register description, then map all that stuff manually to a structure which kinda-sorta copies the idea and layout of the hardware registers, but not quite with subtle differences, and then call a superfluous init function with that struct, which then painstakingly writes those actual registers which you could have accessed as easily. There is zero portability and it abstracts nothing. Functionality is changed, but in a way which offers no benefits whatsoever. This is abstraction for the sake of abstraction itself, driven by middle managers who have no idea about real-world programming but think they know layers are needed because they look good on PowerPoints.

[Perkele]

I am having a lot of difficulty in deciding what sort of I/O abstraction I want to use in my own projects.
something like the "digialwrite()" in arduino, but then something that works properly and with zero overhead (compared to machine code)

When I started uC programming with the Atmel AVR's (10+ years before arduino existed). I used simple direct port manipulation:

Code: [Select]

#define LED_PIN (1<<4)
#define LED_PORT  PORTD

// Turn the LED on:
LED_PORT &=~ LED_PIN;
It works, is simple and fairly portable among different uC families, but it does not look pretty in the software, and I am looking for a method that is more readable in the source code.

To counter this particular example, with "modern" AVRs you can avoid read-modify-write by using a much faster access via GPIO OUTSET and OUTCLR registers.
That solution would need a different macro from the one described here.

With time, these types of libraries tend to sprawl until there's a kitchen sink installed somewhere in a helper library. And later another helper library for the original helper library.

Recently on an STM32 project I had to remove STM HAL calls and their version of "portable" I/O functions and rewrite it into something that resembles CMSIS-based code from 15 years ago.
The firmware was too slow and used too much space in code memory.

A LED is almost always an output (except for those weird corner-cases), a switch or a push-button are always an input.

I also had a good exercise in "where do we stop with abstractions?" on another project. Take a mid-size and mid-complex firmware, and port it as a userspace application onto a PC.
We had to throw-away low-level hardware abstractions and ended-up with what NorthGuy said.

[Doctorandus_P]

To counter this particular example, with "modern" AVRs you can avoid read-modify-write by using a much faster access via GPIO OUTSET and OUTCLR registers.
...
That solution would need a different macro from the one described here.
...

AVR GCC has always been smart enough to optimize those bitwise operations into single SBI or CBI opcodes (if it was for a single bit), so I doubt modern AVR's would be any faster. I also don't use any of the modern AVRs. I got thoroughly annoyed about incompatibilities between different AVRs. From renaming bits in registers for no reason at all (which uglifies library code to make it compatible) to big incompatibilities between AVR's and the Tinies, for example UsART versus USI.

I once looked into the Xmega's, nice functions but the difference was so big, that I could as well look at some other architectures too. The Xmega's also had quite small maximum memories (compared to 512kB+ of others such as STM32 and other ARM) And I bought some "Blue Pills" for fun (when they still cost EUR4 and had real STM IC's on them. Due to some brain damage I am a slow learner and forget stuff easily. Working with a bunch of different uC architectures is therefore also no option for me. It also drives my wish for a simple, quick speedy and clean portable library. It's not an "OCD" thing. I have to keep code clean, or I loose overview, get lost and can't manage to write any code at all.

Recently on an STM32 project I had to remove STM HAL calls and their version of "portable" I/O functions and rewrite it into something that resembles CMSIS-based code from 15 years ago.
The firmware was too slow and used too much space in code memory.

And that is probably what got me started on this thread in the first place. I'm looking more into Cube and the HAL, and I see a lot of complication I don't need or want, and that is in the way of what I want to do.

[nctnico]

Rcently I have been working on some STM32 projects because customers where foolish enough to select these for their projects. There are two major things that bug me: the HAL is over complicated but when I try to replace the HAL, I run into issues with the hardware manual being incorrect where it comes to how the hardware actually works. My strong advise would be to dump the STM32 parts and look at NXP's LPC series ARM microcontrollers. The documentation for those is much easier to read, the hardware works as described and the peripherals are way easier to work with. As an added bonus NXP has re-used many of the peripherals across the entire range. So if you have a 'driver' for an SPI interface on a lower end LPC1100 chip, it will also work on the higher end LPC1700 series.

[wek]

I have to keep code clean, or I loose overview, get lost and can't manage to write any code at all.

Try comments.

JW

[cv007]

If you have to bypass it even once in your project, you are duplicating and maintaining two sets of IO code, which is practically doable, but stinks from the elegance viewpoint which was the only reason to do this.

The gpio pins are probably a unique peripheral- both somewhat simple and also something your code has a big role in timing when it matters. I have duplicated sets of gpio classes in some cases, one 'normal' and a second using templates. Was not really used, but it gave the option to do fast if needed and the option of simplicity when not (passing pins to other classes without having to also make then templates). For something like a bitbang protocol, using the 'fast' version would be an option- the protocol class then becomes a template version to handle the templated pin. The gpio/pin class would be essentially the same, but you are giving all the info needed for the compiler to optimize for each use, inside the template parameters (with the resulting downside that its now a specific type). One could go all templates and get optimal code everywhere, but it becomes a pain to create/use and makes the code harder to read.

simple example-
https://godbolt.org/z/13MYvrvqa
a previous example was just quickly modified from a non-template version, so probably may not be ideal/correct (didn't read the asm real closely, either). It does show it can be relatively easy to get something at least 'faster' for special cases, and if its still not enough then there are still other options (including C, asm).

Just like the C programmer has to deal with not having access to C++, the C++ programmer has some things to consider in the area of fast pin manipulation, if/when needed. Its a small problem that shows up infrequently, so can deal with it as needed.

[asmi]

I found a simple yet effective way of dealing with this - I don't have time for any of this crap. If MCU isn't fast enough for what I need it to do, I simply pick up a faster one, if I need more peripherals - I get an MCU which has got them instead of messing with bit-banging. This is why I love vendor-provided HALs - they allow migration across their products with the minimum amount of pain. R&D time is so incredibly expensive, that unless you're shipping your products in millions, any hassle of writing your own libraries, abstractions, etc. is going to cost more money and time, than just picking up the next in line MCU with all functionality your require out of box. All these obsessions with "beauty" of code, abstractions, etc is usually just a waste of R&D money (unless you do it for fun at home) - the code must me as simple and as "standard" as possible, so that if another developer will have to make changes in that code (or even yourself after a couple of years), he should be able to understand what's going on immediately and not wasting time pondering which exact chemical the original author was under when he came up with that code.

[westfw]

Quote from: Doctorandus_P on Today at 01:19:46 pm

AVR GCC has always been smart enough to optimize those bitwise operations into single SBI or CBI opcodes

On the new AVRs, there are "PORTS" with SET/CLR/Toggle registers that can atomically manipulate more than one but, but they are not in the "fast" IO space, so you have to use ST instructions.  But there are also VPORTS that shadow the same hardware and have the old functionality that ARE in SBI/CBI range.
Just in case you thought that abstracting all of the available functionality was going to simple...

[Siwastaja]

AVR GCC has always been smart enough to optimize those bitwise operations into single SBI or CBI opcodes (if it was for a single bit)

Only when the IO port fits in the addressing range of the SBI, CBI instructions, which used to cause nasty surprises on larger devices where some ports act differently.