Suggestion of methods to attach 8+ rotary encoders to an Arduino

[Sonny_Jim]

Been fiddling around with making a cockpit for Flightgear during lockdown and I'm running into a little problem with rotary encoders.  I want quite a few of them and I'm trying to think of ways I can offload the processing of the encoders to a separate chip so the Arduino isn't having to constantly scan them all.

In an ideal world this chip would:

1.  Monitor each of the encoders and provide data for the main Arduino to be picked up via I2C.  So encoder 1 has moved +2, encoder 2 has moved -2 etc
2.  Be fairly cheap

Does such a thing exist?  Or should I just throw in another Arduino dedicated to reading the encoders?

[BlackICE]

You can use change of input bits status to generate interrupts instead of polling the encoder. 3 bits for use of all functions for each encoder. However I don't think that 24 bits are available to use on common MCUs, only one port of 8 for 2 to 4 encoders if you don't care about the push function.

[Sonny_Jim]

Not totally bothered about the pushbutton as the plan there is to go for a 8x8 switch matrix running on a MCP23017.  Also 8 encoders isn't a hard limit, it's just a 'nice-to-have'.

At the moment I have a few wired up via interrupts on a Pro Micro that's emulating a USB joystick, but I'm finding that unless I dedicated a whole Arduino to the encoders it tends to miss the inputs when I spin them quickly.

Maybe I just use a spare MCP23017 with the 3 bits wired up for each encoder.  From reading around you need to hit <1ms scan times or else you begin to miss encoder pulses, so maybe not.

[Ian.M]

If you are going down that route, you should look at the SPI version of that chip: MCP23S17, as its host interface can run over five times faster than the I2C version.  Its got the same IOC capability as the I2C version, so you can idle waiting for any encoder event, then when you see one, disable further interrupts and poll it every millisecond.  It only takes 4 bytes transferred to poll both banks (assuming you release the SPI bus between polls), which even a Uno should be able to do in a very small percentage of the 1ms you have between polls.  If its state hasn't changed for say ten polls, clear its interrupt, update its IOC registers and you can then stop polling and go back to waiting for an encoder interrupt.

Running eight encoder state machines and eight software up/down counters fast enough not to eat too much of your available CPU time is more problematic.  That alone would be a good reason to handle the encoders in a slave MCU. 

Considering dedicated hardware, if you want to go down a very '80's route, well you can still get QEI to up/down counter interface ICs, of which you'd need one per encoder + some way to select each to read it, the cheapest option there probably being to dedicate a PCF8574(A) per encoder.   The PCF8574 has an IOC interrupt output, so you wouldn't have to poll continuously.  However, that's certainly not cheap or convenient with three chips per encoder @ ~$10, and a large chunk of your I2C address space used.    The modern alternative would be to use a small FPGA, but if you already had the  toolchain and experience to do that successfully, you'd have done it, rather than starting this topic!

[BlackICE]

Depends what else you have the processor doing. If the only interrupts you are processing are for the encoders you should have no problem handling one and probably two. You only have two hands so you can only turn two at the same time. However I've only tested one and never had an issue. However you have to write tight code and have a good algorithm. I tried various methods using a state machine and also a look up table which is use very little processing time. I forgot the timing but I'm pretty sure in and out of the interrupt routine and less than 20 usec on a 16 megahertz processor. I don't use any of the Arduino libraries and write and read the ports directly.

[Sonny_Jim]

Very interesting, thanks.

If you are going down that route, you should look at the SPI version of that chip: MCP23S17, as its host interface can run over five times faster than the I2C version.

Running eight encoder state machines and eight software up/down counters fast enough not to eat too much of your available CPU time is more problematic.  That alone would be a good reason to handle the encoders in a slave MCU.

The only reason I went with the I2C version was that it uses fewer pins on my microcontroller, but I'm coming more and more to the conclusion that I should just chuck in another Uno and use that as a dedicated encoder processor.  I'll try it with the I2C version and see how it pans out.

The modern alternative would be to use a small FPGA, but if you already had the  toolchain and experience to do that successfully, you'd have done it, rather than starting this topic!

Quite.  Maybe it's time to stop playing around with ESPs and Unos and get a small FPGA.  Although that would be project creep and I'm already staring at a bunch of bits from Aliexpress on my table wondering how exactly they will all go together.

My basic architecture in my head is like this at the moment:

Flightgear sim <- UDP -> Esp8266 <- I2C -> Uno

Esp8266 is the gateway and drives the displays/servos/switch matrix.  Luckily a lot of the switches don't need hyperquick response times, it doesn't matter if it takes an extra 10ms to read the 'fasten seatbelt sign' switch, for example.  Same with the displays, a lot of them display static content for the most part.  It just makes sense to me at this point to focus on using the Uno for encoder reads and passing that information back to the Esp8266 via I2C.

Depends what else you have the processor doing. If the only interrupts you are processing are for the encoders you should have no problem handling one and probably two. You only have two hands so you can only turn two at the same time

That is true.  At the moment I've got 3 encoders running off a Pro Micro, and it's lack of interruptible pins is already a problem.  I think the Uno all the pins are interruptible, so I can run 8? (if I did the maths right).

[Ian.M]

Interrupts for encoders are a PITA and can be very problematic.  e.g. a slightly worn mechanical encoder, (or even a supposedly immune to contact bounce optical or hall effect encoder sitting near a transition + mechanical vibration) can easily generate an interrupt storm and steal all your processing time.  Its a far better idea to poll the encoders at a 1ms sample rate,  and run the eight state machines effectively in parallel, which with appropriate RC debouncing for mechanical encoders, should be good for up to 500 counts per second per encoder, without loosing any.  Worst case, if that's not good enough you should be able to push the polling rate up by maybe an order of magnitude, though you may need to put a 20MHz crystal on the ATmega, and will have to be careful to keep your I2C slave event code as light-weight as possible so it doesn't block for more than a fraction of the polling interval.

[Peabody]

The Atmega328P as found in the Uno and Nano lets you use pin change interrupts on all the I/O pins.  These are not as elegant as the "External" interrupts because the interrupt is generated if any enabled pin on the port - any of 8 - changes state.  Then you have to read the port to figure out which pin changed state.  This is not foolproof because bouncing can change the state of a pin between the interrupt event and the time the port is read in the ISR.

Also, the standard lookup table method will trigger an interrupt on each bounce, and there could be dozens on each transition.  So you might need to write special code such that when the first interrupt occurs the ISR disables further interupts on that pin and enables them on the other pin, which should be stable.  That works fine to eliminate bounce interrupts, but needs special handling when you change direction.

You might want to look at the "Arduino" folder of this Github repo:

https://github.com/gbhug5a/Rotary-Encoder-Servicing-Routines

It has sketches that count the number of interrupts that occur between each encoder detent - for both the standard method and an alternate method that selectively enables pin interrupts.  The alternate method produces far fewer interrupts, but also produces more errors.  But note that these sketches were written for one rotary encoder.  Multiple encoders on the same port would be more challenging.

But in theory, if you don't need to support  the push buttons, you have enough pins on a Nano to handle 8 encoders plus I2C and serial.

Edit: Forgot to say - I think I2C would not block anything during transmissions.  The peripheral built into the chip should handle the transmissions in the background, and should have no effect on whatever else is going on - polling or interrupts.  But you would need a circular buffer of messages to be sent so rapid turns of an encoder would not lose any events (although they might be a bit slow in transmitting).

[Ian.M]

Why would you need to queue messages?  400KHz I2C can transfer up to 44 bytes per ms so even with Arduino overheads any reasonable read routine should be 'done and dusted' in no more than one encoder polling interval, unless the master is slow handling the bytes as they are read from the slave. 

Simply add an interrupt line for the slave to alert the host, or have the host poll the slave, to fetch a single byte bitmap of which encoders have moved since the last poll, then, depending on the result, either read all the encoder counters, or address the individual counters that have changed and read them (atomically) resetting the 'updated' flag for that counter.  Assuming 8 bit counters, it can handle up to 127 encoder counts in either direction without ambiguity.  Use 16 bit counters and even a compulsive knob spinner couldn't get it to go the wrong way!

On choice of Arduino for the slave - an Atmega328P based Arduino has (at most) 14 digital pins and 6 digital capable analog pins.  Two of the latter are needed for the I2C bus, leaving 18 pins, of which two are usually reserved for serial comms with the host.  That's just enough for two lines from eight encoders.  However there is no need for a USB serial port which just adds cost, complexity and unwanted loading on digital pins 0 and 1.  Therefore I suggest an Arduino Pro Mini clone, or better a bare Atmega328P + crystal, load caps and RC reset circuit, and using D0 (PC0/RXD) for the 'encoder change' interrupt to the master.  That leaves D1 (PC1/TXD) free for printf style debugging, if you initialize the USART for TX only using raw AVR code.


 

[rstofer]

Apparently, you can have 24 Interrupt On Change pin interrupts.  Just enough for 8 encoders with pushbuttons...

I would probably use an Arduino Nano.

Here is some logic for handling a single encoder.  You need to figure out how to handle several using the same ideas

http://blog.templaro.com/reading-a-rotary-encoder-demo-code-for-atmega328/

Now that you have the count for the 8 encoders, you need to come up with a scheme for getting that information to a higher level.  I2C is kind of nice because you can have many Nano style gadgets but SPI is easier.  SPI has the disadvantage of requiring a select pin for each device.

With I2C, you would write a single byte to address the appropriate Nano and then read 3 consecutive values of encoder position information.  You could rig the Nano to interrupt the host when encoder data changes.  That's probably a good idea.

The big deal in de-bouncing is in the state table.

[rstofer]

If I was thinking in terms of UDP or TCP, I would probably use an mbed LPC1768 and the libraries available at mbed.org (now owned by ARM).  I would put as many of these on the network as I wanted and they could just send and receive data all day long using Berkeley Sockets.

Just build a daughter card for the actual IO connections and you're good to go.  The attached example acts as an interrupt driven SPI slave, grabs the data, converts it to blocks and sends it out over TCP to a LaserJet.

Depending on how I felt in terms of cost, I might use 1 assembly per encoder although it would be easy enough to handle several.

I would probably try to base the entire project on UDP because once I got it working, adding more devices is easy and each can be wildly different than its neighbors.  Set up a socket at the device, connect to the host and start talking.  Asynchronous interrupts of the host would be a great deal more difficult but message passing will work well enough.  Testing is easy, use a terminal emulator like PuTTY to talk to the end devices.  Try to pass messages in an ASCII format rather than pure binary.  That makes testing a lot easier.

Maybe have several kinds of messages.  Binary for raw data in production, ASCII data for use with PuTTY and HTML for use with a browser.  There's no reason I can think of where you couldn't just hit a specific port to get the data in a specific format.  Port 0 for binary, Port 23 for Telnet (ASCII) and Port 80 (or wherever) for HTML.  The device could accept one or more connections on each socket.

Get one device to work and use it as a model for all the others.

Of course, the CAN bus was invented for this kind of application.  I have never used it...

[Peabody]

The 328P has three I/O ports, but some pins are dedicated to other functions, or just not brought out in the Nano.  So assuming  you need pins for serial and I2C, you have 16 pins left that could be used for encoders - A0-A3, D2-D7, and D8-D13.  If you don't need serial, then D0 and D1 could also be used, and you could support nine encoders, but not their push buttons.

It's not obvious to me how the firmware would work for this, particularly given the fact that multiple pins could change at the same time.

[Ian.M]

It's not obvious to me how the firmware would work for this, particularly given the fact that multiple pins could change at the same time.

Poll all the ports and shift and mask into a 32 bit state vector of previous state interleaved with current state in two bit slices.  Each four bit chunk can then be decoded to determine if each counter needs to increment, decrement or stay the same, so grab a copy of the state vectors, and loop through the counters, extract the low nibble to determine the action, and shift the state vector right four bits to position it for the next counter on the next pass through the loop.   The only thing you need to handle specially is ignoring any transition where both encoder lines have flipped since the last poll - that's an illegal transition indicating a bounce or intermittent contact problem, or severe over-speed - so copy the old two bits to the new ones to discard the bad data  and increment a (per encoder?) error counter.  If the problem was bounce it will probably 'come good' on the next poll, but overspeed and persistent bad contact will be detectable by the rapidly incrementing error count.

[rstofer]

The 328P has three I/O ports, but some pins are dedicated to other functions, or just not brought out in the Nano.  So assuming  you need pins for serial and I2C, you have 16 pins left that could be used for encoders - A0-A3, D2-D7, and D8-D13.  If you don't need serial, then D0 and D1 could also be used, and you could support nine encoders, but not their push buttons.

It's not obvious to me how the firmware would work for this, particularly given the fact that multiple pins could change at the same time.

I don't think it matters, you will only clear one interrupt on each pass through the interrupt handler.  An unhandled pin change on one pass through the handler will still cause an interrupt as soon as the handler returns.  You could get fancy and handle multiple changes.

I haven't done this on the ATmega328P but on the ATmega128, it is possible to designate whether the interrupt responds to the rising or falling edge and, having seen a rising edge interrupt, we change the mask such that the next pin interrupt will occur on the falling edge.  If we get a nasty transition on the rising edge, it doesn't matter.  We're looking for a falling edge.  If we do get a bounce (invalid transition), the state machine takes care of correcting the count.

Nanos are cheap, it is entirely possible to use 1 Nano per encoder.  Keep it simple!

[rstofer]

Here is a nice way to reduce the problem from quadrature decoding to simple step/direction inputs to the host.  We only need to interrupt on the step and read the direction.  I could see building a bunch of these interface boards, one per encoder.  Or, several on a single PCB...

https://www.bristolwatch.com/ele2/rotary.htm

[Ian.M]

Any QEI circuit with a single flipflop clocked from one of the quadrature signals to determine the direction tends to 'run away' if that signal bounces or is otherwise jittery.  The direction doesn't change as the other signal is stable, so every pulse from bounce or jitter increments the counter in the same direction even though the encoder is at most rocking to and fro slightly.

To do it right in hardware you need to build something like the state machine that better software implimentations usually use.

See my comments from four years ago here:
https://www.eevblog.com/forum/projects/quadrature-rotary-encoder/
which also has a LTspice sim I posted of a hardware QEI decoder implimenting the standard state machine.

[spostma]

Many years ago I made an efficient decoder in Microchip PIC16 assembly that decodes 4 digital potmeters per byte,
and as many of those bytes as you like.


There is one include file variant with 8-bit counters and one with 16-bit counters.

The byte with 4 pometer signals is XORed and shifted in parallel for 4 biphase signal pairs in one operation.

I am sure you can take this idea and port it to your processor.

[BlackICE]

Here is some sample code I tried and it worked very well with the cheap encoders I got from Amazon. I never detected any errors while turning as fast as I could. YMMV depending on the encoder. You can try both the state based or table driven code by commenting out the unused section of code. Of course the code would have to be modified for multiple encoders, but this will give you as base to start from.

As for processing the interrupts you should process all changed bits (all encoders on the port) before exiting the interrupt routine. View the interrupt routing as a asynchronous polling loop only call when actually needed instead of wasting cycles polling when most of the time nothing has changed.
 

/*
 * Use PORTB bits 0 and 1 as S1 and S2 of rotory encoder. We use change if pin interrupt to
 * find out which direction the encoder is moving.
 *
 */
 
#include <avr/interrupt.h>

typedef signed char S8;
typedef signed int S16;
typedef signed long S32;
typedef unsigned char U8;
typedef unsigned int  U16;
typedef unsigned long U32;

typedef struct // Rotory_En
{
  U32 etime;
  S16 counts;
  S16 errors;
  U8 button;
}
Rotory_En;

#define SV 2    // starting value, fudge to get divide by 4 to work

//static volatile int button = 0, count = 0, errors = 0;
static volatile Rotory_En encoder = {0,SV,0,0};

ISR(PCINT0_vect)
{
  unsigned char pb;
  static volatile U8 lc = 0b1100; //fudge init number to get correct value most of the time for the 1st int after startup
  unsigned long startt;
 
  static S8 delta[] = {
    0,  // b0000 button or error
    1,  // b0001
    -1, // b0010
    0,  // b0011 error
    -1, // b0100
    0,  // b0101 button or error
    0,  // b0110 error
    1,  // b0111
    1,  // b1000
    0,  // b1001 error
    0,  // b1010 button or error
    -1, // b1011
    0,  // b1100 error
    -1, // b1101
    1,  // b1110 error
    0   // b1111 button or error
  };
 
  startt = micros();
  pb = PINB;      //read port b
  encoder.button = (pb & 0B100) / 4;
  lc = ((lc * 4) | (pb & 0x3)) & 0xF;
  encoder.counts += delta[lc];
 
/* 
  switch (lc)
  {
    case 0b1110:    // clockwise
    case 0b1000:
    case 0b0001:
    case 0b0111:
      encoder.counts++;
      break;
    case 0b0100:    // counter clockwise
    case 0b0010:
    case 0b1011:
    case 0b1101:
      encoder.counts--;
      break;
    case 0b1111:    // if this then just button pressed
    case 0b1010:
    case 0b0101:
    case 0b0000:
      break;
    default:
      encoder.errors++;
  }
*/

  encoder.etime = micros() - startt;
}

void setup()
{
  Serial.begin(115200);
  //delay(100);
  Serial.print("Starting ");
  Serial.println();
  cli();
  DDRB &= B11111000;   // set port B bits 0, 1 and 2 as input
  PCICR |= 0x01;       // turn on port B change interrupts.
  PCMSK0 |= 0x07;      // setup port b pins 0, 1 and 2 to intterupt on change
  sei();
}

void loop()
{
  static int c = SV;

  if (c != encoder.counts)
  {
    Serial.print(" Counts = ");
    Serial.print(encoder.counts / 4);
    Serial.print(" Errors = ");
    Serial.print(encoder.errors);
    Serial.print(" Button = ");
    Serial.print((S8)encoder.button);
    Serial.print(" Time = ");
    Serial.print(encoder.etime);
    Serial.println();
    c = encoder.counts;
  }
}

[Ian.M]

The table is just another way of implementing the same state machine algorithm, with poorer (i.e. nonexistent) error handling than the switch()/case version.

[BlackICE]

The table is just another way of implementing the same state machine algorithm, with poorer (i.e. nonexistent) error handling than the switch()/case version.

Yes, you are correct!

What version to use depends the quality of the encoder and the use of the encoder. I just bought some to experiment with, never making anything yet using one. My uses would just be for personal use as a input device where if it screwed up counts would be a nuisance not a product liability. Depending on the compiler used, one version may be more CPU efficient that the other. When I cared in the past I would look at the assembly generated and count cycles to optimize the code. eg. many times shifting by powers of 2 is faster than dividing, you want to use registers whenever you can and avoid memory accesses, etc. This is hard to control using a C complier. If I cared enough and/or the complier was poor I would write it in assembly code. I suspect an optimized table version would be faster at the expense zero error detection.

This is a photo of the encoder I used. It has 5 resistors and 2 caps on the backside. I don't even know what type it is but I got zero errors while testing. That's why I wasn't too concerned about errors for personal use.

[Doctorandus_P]

I did not read all posts...

I would use another microcontroller to monitor the rotaries.
If you give such a (20 or so pulses /rev) rotarie a good swing, then  the flanks can be less then a milli second apart (There are 4 flanks for each "pulse") so you would preferably sample them at a bit higher frequency, maybe between 5kHz and 10kHz.

If you want to sample 8 of such encoders at 10KHz with a microcontroller that runs at 8MHz (a common RC oscillator frequency) then you only have 100 CPU clocks to sample each encoder and turn the changing flanks into a pulse count. It's doable in software, especially if you're willing to miss some pulses, but you should spend a bit of time to streamline your software. Prices of small microcontrollers start at 50ct or so, (at least before covid happened) and most uC's have both I2C and SPI slave functionality, so you can quite easily turn them into a smart peripheral.

Alternatively, you could use for example an STM32. These have hardware support for keeping track of several rotary encoders at once. These are designed for keeping track of motor encoders, which can have a few thousand of pulses per revolution and rotate at 3000rpm. Common pulse frequency is around 100kHz, but it's handled in hardware, so the microcontroller can handle several pulse streams in the MHz range each.

[Sonny_Jim]

Thank you for all the indepth reponses, a lot to think about.  I don't mind missing 'a few' pulses, as long as the 'feel' of the thing works out alright.

The way I was handling them before (and getting poor results with) was wiring up the encoders to a Pro Micro acting as USB joystick, which the flightsim then picks up on.  The problem with that approach is that the flightsim will miss very quick button presses.  Fortunately it has a very open API, so the plan is to remove all off the 'dial reading' from the flightsim itself and get that data from the Arduinos.  So rather than passing to the flight sim 'Ok Flightsim, here's a quick load of button presses, count them all up and update the autopilot altitude', I'm processing everything on the Arduino and saying 'Ok flightsim, I've counted the rotary pulses and the new autopilot altitude I'd like is xyz'.

[David Hess]

The AVR processor used in the Arduino is so inexpensive that I would consider using them to monitor the encoders and communicate back to the Arduino with a serial link.

[Sonny_Jim]

Yes that was the approach I took and it's working out just fine.

6 encoders going into an Uno which then sends the position deltas to the 'main' ESP8266 CPU via I2C.  Even if I spin the encoders fast it doesn't seem to skip/slow down.

Also got a 8x8 switch matrix working via a MCP23017 so I should have plenty of inputs (for now).

[Renate]

Its a far better idea to poll the encoders at a 1ms sample rate...

What he said ^

Just take the incoming Gray code and convert it to binary and subtract your current known position, that is your delta.
What about errors? There are no errors. What if it the delta is +/-2?
Acceleration is not infinite, you use your state variable for direction to resolve it.