I'd vote for the Rigol unit also. Although you might want something faster in the future, for 95% of stuff, that little scope will do the job for you and will probably still be useful in many years to come, even if you one day buy a faster one.
Regarding your encoder getting dodgy when you speed it up, a couple of things to consider:
1) are you sampling the pins in some kind of polling-loop? (i.e. some bit in your code which keeps checking if the pins have changed state?) If so, this might be too slow, or may have times when polling is delayed due to other operations. Consider either polling in a timing interrupt, or if you can, use interrupt-on-change for those pins
2) If it's a rotary encoder, to get the best performance, you need to carefully consider the switch-bounce that happens during transitions, and how your code interprets what an increment/decrement condition is. In general, you should look for a two-stage pattern, not just that one pin has changed state. When I've made these types of things in the past, I wrote them as a state machine in software, so at any given state, what's the possible next-states. And only when you transition through TWO states, can you really consider it an increment or decrement - otherwise, you can end up trying to follow a switch bounce. To test if you really did this properly, set up a number that will increment/decrement that you can read somehow (either debugger, or your display or similar), then grab the knob (with it securely fixed!), noting your 'starting position' and violently twist it back and fourth for a while. When you return the knob to the starting position, the number should be back to the starting number (usually zero). It's surprising how much kit/commercial products don't do this - and slowly the number/display shifts one way or another - sometimes called 'knob creep'.