From a newbie
I started with used analog scopes first and out of 3 all 3 had problems, one busted in the mail, one was busted, and the third one has so many problems I hate getting near it. Lost a lot of money trying to get an used analog scope, so I would suggest do not try. But if you like to gamble, go for it.
The fourth and five scopes were a Rigol DS1054 and a DSO 112 and I like them both, and I use the DSO 112 more just because it amazes me and is very "cute" and I use the Rigol to keep the DSO 112 honest (and so far it has been). I would be happy to have just the DSO as a beginner. For example I can see the ripple from a buck converter on the DSO 112 but not the high freq signals on the ripple (need the Rigol for that). You can do a lot of learning with a 2 meg bandwidth. If you get a DSO 112 make sure it has a battery, most do not.
I also took advice on getting used Power Designs power supplies, and got 2 out of 3 good ones. I like their look better than new crap. I could suggest going this route. I think you could get to 90% good luck if you ask the seller to turn the voltage dial and see if the meter moves. But this takes time, look at ebay SOLD listings to get the values, not Buy It Now prices.
If you want to go cheap on a ps, try a laptop supply and some buck converters. Hard to beat them and you can learn a little, this is what I am doing.
I went for a couple $50 multimeters (found on the multimeter spreadsheet - I wanted ones that use AA batteries) and am happy with them. I use cheap Harbor Freight meters for current measurements because I am a newbie and I do not want to blow up my $50 meters when measuring currents.