UI design - using small joysticks instead of rotary encoders

[SiliconWizard]

This is something I've been considering for a while, but I'm curious to know what you think of this approach, if you have done it, the drawbacks, etc.

So, that would be using small analog joysticks instead of rotary encoders, typically for navigating menus and changing parameters.
The typical kind of joysticks I have in mind (at least for the smaller controls) are these like in the PS5 game controllers (Hall effect- based). Should be reasonably robust.

My typical idea of UI with these would be something like the following:
- Up/Down: change value. Since they are analog, the idea would be that the more up or down we push them, and the faster the value changes.
- Left/Right: similarly, but rather for changing menu items / parameter type.
In both cases, a simple push from the zero position would just increment or decrement once, and would start auto-incrementing/decrementing (at a speed depending on position) only after a given delay - something like half a second. So that would avoid having values changing by several increments if one doesn't mean to.

That appears to be more efficient to use than rotary encoders, while still being rather intuitive.
One minus point I see though is that, depending on how they are positioned on the device in question, they may require using one's fingers in a way that's not as comfortable as just rotating a knob (in particular on a vertical panel).

What are your thoughts? UI design is not something completely obvious.

[SteveThackery]

There are countless examples of this kind of UI. Every TV remote has a four-way pad with a central 'OK' button, which is logically equivalent to a joystick, but much cheaper. You can observe how the UI works for one of those. In general they don't work in the way you describe - the central "push" doesn't change any values, but rather locks in the selected value and goes back one menu layer.

[tooki]

This is something I've been considering for a while, but I'm curious to know what you think of this approach, if you have done it, the drawbacks, etc.

So, that would be using small analog joysticks instead of rotary encoders, typically for navigating menus and changing parameters.
The typical kind of joysticks I have in mind (at least for the smaller controls) are these like in the PS5 game controllers (Hall effect- based). Should be reasonably robust.

My typical idea of UI with these would be something like the following:
- Up/Down: change value. Since they are analog, the idea would be that the more up or down we push them, and the faster the value changes.
- Left/Right: similarly, but rather for changing menu items / parameter type.
In both cases, a simple push from the zero position would just increment or decrement once, and would start auto-incrementing/decrementing (at a speed depending on position) only after a given delay - something like half a second. So that would avoid having values changing by several increments if one doesn't mean to.

That appears to be more efficient to use than rotary encoders, while still being rather intuitive.
One minus point I see though is that, depending on how they are positioned on the device in question, they may require using one's fingers in a way that's not as comfortable as just rotating a knob (in particular on a vertical panel).

What are your thoughts? UI design is not something completely obvious.

I think it’s an interesting idea, but not a good one.

For starters, some people (like myself!) just struggle with analog joysticks. I can’t say for certain why, but I have never managed to get the hang of the analog sticks on game controllers. I suspect the extremely low force they take to move might be part of it. The lack of haptics is probably a big part of it. Maybe if they had more resistance I’d find them easier to use. (I have no trouble whatsoever with real-world analog controls, like a car’s steering wheel.) And if I — someone with normal, unimpeded motor skills — can have trouble with them, they may prove extremely difficult for people with certain disabilities.

A more fundamental issue I see is this: the type of menu structure you describe is very common, but it also has the interaction pattern that you are always interacting only with one axis at a time: one axis to change between parameters, another to select the value. So what would happen if you used a standard analog stick that allows fluid motion on both axes at the same time? Do you continue changing values even as you simultaneously scroll between parameters, changing things you didn’t mean to change? Do you artificially ignore or prioritize one axis? If so, what are the control dynamics you decide on? Or do you buy/create an analog stick that allows only movement on one axis at a time (for example, by having a +-shaped gate for the stick itself, so that once you move into one “arm” of the cross, your movement is limited to that axis), much the way the original NES and Game Boy are designed to make it practically impossible to press two axes at once (like up and right), in contrast to models with circular D-pads that support a “northeast” press.

If there were some key advantage to using analog sticks that way, and no serious downsides, we’d probably see them.

[ajb]

I'm not a huge fan of rotary encoders for menus in general because of the limited number of inputs -- a good interface of any complexity needs at least four inputs IMO, so you can have a back/cancel as well as up, down, enter -- so a 4/5-way directional pad would be my goto.  But I agree with tooki about the usability of an analog joystick.  If you need to support a lot of interactions that are both 2-axis and analog in nature, like jogging a 2-axis machine or something -- then it could be good for that, but I'd say it's a better addition to conventional buttons if you also need to navigate a menu with more discrete interactions.  You could process the input to reject ambiguous/probably unintended inputs, but that has opportunities to introduce other annoyances for the user if they're not designed carefully, and in any case you're throwing away most of the information the device was designed to provide you with.  While the ability to control speed of scrolling through options or parameter values could be nice if there are a LOT of parameters or very large ranges, I think the more conventional button-based solutions to that work better for the vast majority of menus. 

[Benta]

Analog joysticks are an excellent choice for mnay things.
For example controlling RC aircraft, where you have an immediate feel for what your joystick input does.
But for menu navigation? Nah. If you want to get the feel for that, just take a laptop and use the rubber nipple in the middle of the keyboard.
Useless, really.
And you still need some kind of selection button (FIRE button on a game controller perhaps?)
IDK.

[floobydust]

A UI is typically doing two things that must not be conflated.
You are navigating a menu system, tree-structured. Need to move left/right, maybe up/down into a branch or back up out of it.
Then, to change a parameter/number setting you need to select and incr/decr the value without screwing up and leaving the parameter menu, branch included.

A rotary encoder with a push button is gross- you can spin through the menus, press to enter the menu item, spin to select a parameter, press to enter the param and spin to change it. Now, how the hell do you back up and get out of this?

I'd say you need two pushbuttons - enter and back. Which is almost like having 4 buttons in total, my LCD monitor does this and I hate it.

[timeandfrequency]

A UI is typically doing two things that must not be conflated.
You are navigating a menu system, tree-structured. Need to move left/right, maybe up/down into a branch or back up out of it.
Then, to change a parameter/number setting you need to select and incr/decr the value without screwing up and leaving the parameter menu, branch included.
[...]

This small device might be useful. Shown quickly by Dave here.

[Siwastaja]

So what would happen if you used a standard analog stick that allows fluid motion on both axes at the same time?

That's the key issue I think. The whole point of 2D joystick is, it's 2D. It's for controlling something which is 2D by nature. SiliconWizard's issue however is not 2D, it is 2 x 1D which is totally different; the axes must be controlled separately as their combinations do not make any sense.

By forcing 2D controls over 2x1D, not only you waste (lim)100% of the control surface (everything which does not lay directly on the two axis), you also need to ignore that extra information by generating some mapping into the axes (like a nearest neighbor function); and then the user needs to be careful to use the joysticks in the right position because they won't get the natural feedback of where they are pushing the sticks, and may accidentally go too far to the unintended axis, i.e., change menus when they want to change values. And this is specifically difficult with small movements: are you going to slowly change the value, or slowly switch the menus?

Joystick however works really well for e.g. controlling a robotic arm where the 2D joystick directly maps to two full degrees of freedom, and user sees where it is going, therefore feeding back the position of their fingers on they joystick to get the desired movement.

[madires]

Many devices have up/down/left/right buttons. But they are simple push buttons. My guess is that an analog joystick would cause confusion as it doesn't correspond well with simple UI interactions like decrease/increase.

[Siwastaja]

All that is really needed is to spend a little bit of effort in fine-tuning the auto repeat delay plus auto repeat rate increase function shape, with real testing.

After such optimization, 4-button left/right/+/- interface is excellent; most that suck, only suck because manufacturer uses very poor quality pushbuttons or/and completely failed optimizing the autorepeat user experience.

[tooki]

All that is really needed is to spend a little bit of effort in fine-tuning the auto repeat delay plus auto repeat rate increase function shape, with real testing.

After such optimization, 4-button left/right/+/- interface is excellent; most that suck, only suck because manufacturer uses very poor quality pushbuttons or/and completely failed optimizing the autorepeat user experience.

Totally agree. A good acceleration algorithm looks and feels natural and transparent.

What’s frustrating are poorly designed acceleration algorithms. There’s one brand of industrial PID controllers that we have a bunch of at work, and while they actually work really well as controllers, the UI acceleration is super annoying, because it triggers at the wrong times and doesn’t “release” quite when one expects. E.g. temperature set points: holding it first goes in 1° steps, so 0-1-2-3-…-11 and then moves to 10° steps, so …21-31-41… (A slightly smarter algorithm would count to 9 in 1° steps, then continue with 10-20-30-40…)  Let’s say you want 90°, so you hold it till 91° and let go. So now you have a residual 1° so you press “down” once. You’d assume that because you let go, the acceleration has reset, so you expect it to go down 1°. But the acceleration uses a timeout timer, and it hasn’t run out yet, so instead it subtracts 10° and you’re back at 81°. You have to wait a second or so before you can move in 1° steps again.

Are those ultimately little things, in the grand scheme of things? Yes. But they’re so easily avoided with a bit of effort. I’m not in any way a gifted MCU programmer (heck, I’d lean more towards… mediocre), but I’ve gotten some pretty nice button acceleration working well. If I can do it, anyone can…

[Siwastaja]

What’s frustrating are poorly designed acceleration algorithms. There’s one brand of industrial PID controllers that we have a bunch of at work, and while they actually work really well as controllers, the UI acceleration is super annoying, because it triggers at the wrong times and doesn’t “release” quite when one expects. E.g. temperature set points: holding it first goes in 1° steps, so 0-1-2-3-…-11 and then moves to 10° steps, so …21-31-41… (A slightly smarter algorithm would count to 9 in 1° steps, then continue with 10-20-30-40…)  Let’s say you want 90°, so you hold it till 91° and let go. So now you have a residual 1° so you press “down” once. You’d assume that because you let go, the acceleration has reset, so you expect it to go down 1°. But the acceleration uses a timeout timer, and it hasn’t run out yet, so instead it subtracts 10° and you’re back at 81°. You have to wait a second or so before you can move in 1° steps again.

Bullshit like this gives a feeling that manufacturer either does not care about user experience at all, or the design is very rushed or seriously resource-constrained; or both.

Fixing the UX is such tiny part of total engineering cost. Whoever writes the firmware can do it decently in an hour or two. At company level all that is required is a down to earth manager who says "hey, this is crap, you obviously see why, please fix, thank you in advance".

[PlainName]

A rotary encoder with a push button is gross- you can spin through the menus, press to enter the menu item, spin to select a parameter, press to enter the param and spin to change it. Now, how the hell do you back up and get out of this?

Separate back button. Should only be needed rarely - the encoder press will select from menu and/or confirm value. If the value hasn't changed then confirming will be an exit, and if it has changed then that's when you need the specific abort button.

For menus, you can have an extra item: up/back, just like your nav bar in your filemanager has (that, admittedly, I never use).

I'd say you need two pushbuttons - enter and back. Which is almost like having 4 buttons in total, my LCD monitor does this and I hate it.

As above, just the one which you probably won't touch anyway.

[tooki]

What’s frustrating are poorly designed acceleration algorithms. There’s one brand of industrial PID controllers that we have a bunch of at work, and while they actually work really well as controllers, the UI acceleration is super annoying, because it triggers at the wrong times and doesn’t “release” quite when one expects. E.g. temperature set points: holding it first goes in 1° steps, so 0-1-2-3-…-11 and then moves to 10° steps, so …21-31-41… (A slightly smarter algorithm would count to 9 in 1° steps, then continue with 10-20-30-40…)  Let’s say you want 90°, so you hold it till 91° and let go. So now you have a residual 1° so you press “down” once. You’d assume that because you let go, the acceleration has reset, so you expect it to go down 1°. But the acceleration uses a timeout timer, and it hasn’t run out yet, so instead it subtracts 10° and you’re back at 81°. You have to wait a second or so before you can move in 1° steps again.

Bullshit like this gives a feeling that manufacturer either does not care about user experience at all, or the design is very rushed or seriously resource-constrained; or both.

Fixing the UX is such tiny part of total engineering cost. Whoever writes the firmware can do it decently in an hour or two. At company level all that is required is a down to earth manager who says "hey, this is crap, you obviously see why, please fix, thank you in advance".

Alas, many engineers are “blind” to UX, because it makes sense to them through the lens of understanding the inner workings, and many people in general simply aren’t attuned to such things. When someone like that is in management, UX gets ignored.

[SiliconWizard]

A UI is typically doing two things that must not be conflated.
You are navigating a menu system, tree-structured. Need to move left/right, maybe up/down into a branch or back up out of it.
Then, to change a parameter/number setting you need to select and incr/decr the value without screwing up and leaving the parameter menu, branch included.
[...]

This small device might be useful. Shown quickly by Dave here.

This is an interesting compromise, although it's relatively expensive.

[timeandfrequency]

This is an interesting compromise, although it's relatively expensive.

Slightly cheaper, but not sure if those are genuine.

[Buriedcode]

Since menus only require up/down/left/right, in discrete steps, it doesn't make much sense to use a system that can have combinations, like diagonal.

You can get "nav switches" that are essentially mini joysticks, but only work in the four directions, anywhere, even the micro ones that were used on old blackberry phones.  Some have a definate tactile "click", others are much smoother, feels more like rubber membrane switches (I'm assuming these are padded internally).

For fascias/front panels were size is a contstraint I've used these, which allows the used of direct buttons, an "enter" and a jog wheel for entering values that vary by a lot and would require a lot of "clicking" to set. They're not great, and without an extra "back" button, one has to add "return" at the bottom of every menu, but they're quite useful for a the few DMX products I've made, and only require
a single cutout for 5 buttons + encoder.

https://www.tme.eu/en/details/ano-i1b-m1b-02b-sz/incremental-type-encoders/zippy/ano-i1b-m1b-02b-s-z/

[timeandfrequency]

[...]
https://www.tme.eu/en/details/ano-i1b-m1b-02b-sz/incremental-type-encoders/zippy/ano-i1b-m1b-02b-s-z/

Interesting too these nav keys, if a flat front panel is required.
However, the part's no longer available from TME.
I found this one having a similar shape, at least on the picture, but not sure if it's a 1:1 equivalent. 

[thm_w]

The ALPS one looks interesting, but large and complex. LCSC has many: https://www.lcsc.com/products/Multi-Directional-Switches_434.html
This is probably the common one used everywhere: https://www.aliexpress.com/item/4000681535647.html

I think an analog joystick could work if it had detents, and if you physically limited the movement to X and Y only, as mentioned. Would it be better than a simple 5-way control, or rotary encoder + two buttons, probably not?

[SiliconWizard]

The ALPS one looks interesting, but large and complex.

The point is that it's a combined encoder and switches. Multi-directional switches only are less practical for quickly adjusting values.

I think an analog joystick could work if it had detents, and if you physically limited the movement to X and Y only, as mentioned. Would it be better than a simple 5-way control, or rotary encoder + two buttons, probably not?

I'm going to experiment a bit to see what can be done.
Properly using it may require some (relative) "dexterity", which not everyone may have, as tooki pointed out.
I don't think the fact it can go in both directions at the same time is a real problem. Pushing it in both directions at the same time requires more force and thus is not as likely to occur by accident as some may think. But that'll have to be tested. OTOH, in some modes, having both axes could be put to good use (if there is a 2-dimensional parameter of some kind).

The real downside IMO is twofold: first, being a bit unusual, so requiring some getting used to. A simple rotary encoder or +/- buttons, everyone knows how to use that. And second, as I mentioned, it may not be very comfortable to use vertically, and more adapted to horzontal panels/consoles or handheld stuff.

And, in any case, you usually need at least ENTER and BACK buttons, so those buttons would still be needed.

I have seen combined rotary encoder + switches, smaller and cheaper than the ALPS one and more adapted to handheld devices as well. The rotary part is usually flat and meant to be actuated with a thumb. So, not ideal for anything not handheld. Yes, something that looks like the last one timeandfrequency posted, although I've seen some that are a bit larger and probably more comfortable to use. But still, those are typically "flat".

[Siwastaja]

Pushing it in both directions at the same time requires more force

It really depends on the joystick. Since 2D joysticks are usually designed for 2D input, uneven force (depending on direction) is an undesired shortcoming which makes the intended 2D use more difficult, so with well designed joysticks the spring return force only depends on the length of the vector (how far you are pushing the stick), not its direction. Then again, some applications are 2D input but with "tack" over the axes desired and then such extra force when combining axes can be even desirable. So you would need testing with actual different manufacturers and models.