Hi
As with all questions the real answer is going to be "that depends".
For something like a 10 turn pot, it's rare to need anything other than a normal screwdriver.
For things like a half turn "open frame" pot where the adjustment slot is directly connected to the pot wiper ... yes a plastic or ceramic tool will save some grief.
You mention scopes, so there is another category. Stuff that has high voltages on it can be tricky. A pot that has 2.3KV on all it's terminals is going to be hazardous regardless of it's construction. Unless it has a grounded shaft (like a chassis mount part), use a ceramic tool on it. The same caution (watch out for high voltage) is a generally good one to remember if you have grown up with low voltage stuff. Back years ago (in the days of tubes), getting killed working on circuits was not at all uncommon.....
Bob