Power Supply, you say... Well, I've been messing around with electronics for 60 years or so and I still don't have one. Don't have a decent signal generator either! I recently bought a DS1054Z scope to supplement (not replace) my Tektronix 485 that I bought 12 years ago for $200. I have had other incantations since I built one when I was about 10 years old from plans in the ARRL Handbook. A scope is required!
How ever do I get along without a power supply? Most of my projects are microcontroller based and those plug in via USB or 5V wall wart. I use batteries for small robotics projects and I use wall warts to feed 7805 regulator type devices. Yes, I'm going to buy a Rigol DP832 one of these days.
I can generate a lot of square waves with a microcontroller, CPLD or FPGA so I don't spend much time missing a proper signal generator. Nevertheless, I plan to buy the Rigol DG1032Z at some point.
I can't say much about LTAOE, I haven't worked through the book although I plan to. I bought the recommended Global Industries PB-503 Prototype Board but it is really expensive. It does have 3 power supplies, a signal generator (sine,triangle,square) plus TTL along with switches, knobs and dials. There is a generous breadboard area. I don't know what to think about the tool but I suspect it will be very handy. There are clearly other ways to do this.
Why buy this stuff at my age? Well, my grandson just started EE school! I hope he will get a lot of use out of these tools over the next few years.
I almost always recommend that newcomers get the Digilent Analog Discovery device because it has two scope channels (full differential, very interesting!), two signal generators (arbitrary waveform), two power supplies (very limited current and voltage), 16 bits of digital IO which can be used as a logic analyzer as well as simple IO and some neat features like a network analyzer. FWIW, the network analyzer scans across a frequency range and plots the output of the circuit versus frequency. Handy for filters and such.
For $279, there a lot of electronics that can be done with that gadget. Serious electronics! This is not a toy!
http://store.digilentinc.com/analog-discovery-2-100msps-usb-oscilloscope-logic-analyzer-and-variable-power-supply/If I were just starting out, I would seriously consider the Analog Discovery (and, yes, I have one) and scale the circuits in terms of voltage such that the AD could handle it. There are limits to the device and they need to be heeded. There'll be no measuring mains voltages or higher voltage ampllifier circuits. Troubleshooting PC power supplies is pretty much out of the question (or at least it is an advanced topic).
There are junk USB scopes out there and there are also some very nice tools. The AD is one of the better devices. There's nothing quite as nice as a 27" scope screen!
Just starting out, get a decent DMM and the Analog Discovery. That lab is complete and capable for a long time to come.
ETA: The power supply capabilites of Ver 2 are seriously expanded. It can also be used with an external wall wart to provide up to 750 mA (max 2.1W) per output. And the output is variable.