Be aware there are five major sub-disciplines which overlap at their edges (analogue, digitial, RF, software, high voltage/current electrical). It is rare to find someone that is competent in all those, but everybody should have an appreciation of all of them, so that they can know what they don't know.
Based on your background and the theory you already understand and like, you sound like an engineer rather than a hobbyist.
It sounds like you would benefit from a three-pronged approach:
- a "30000ft" introduction into the landscape, so that you can have a good idea of where to start on a specific project
- once you know what you are going to do, a rapid dive down into some solid theory and practice for the relevant topics
- practical projects to cement the above and keep you entertained
For (1), there are a number of "beginning electronics" books that people on this forum recommend. I'd have a look on Amazon and see which might suit you.
For (2), read manufacturers' data sheets and application notes. The best book at this level is without doubt "The Art Of Electronics". While its objective is to allow physicists to do electronics, it is far more widely useful than that. Highly recommended; I got the 1st edition when it came out, and have added the recent third edition.
For (3), that's up to you
You will need some equipment, but (with imagination and understanding) less than many people would have you believe. The essentials are:
- some means of supplying power; a fixed voltage mains power supply is sufficient for each project, a bench power supply more flexible
- a soldering iron etc for wires and through hole components; don't be afraid of SMD components, but they require more skill (see blog in my .sig for my experiences). Avoid solderless breadboards
- some means of measuring voltage and current. For digital circuit outputs and inputs LEDs are ideal, but a handheld DMM is invaluable. Be aware of safety limitations
- some means of applying signals. In many cases a switch and potentiometer can be sufficient; beyond that it will depend on whatever your project requires
- if budget and your time allow, an oscilloscope and signal generator. If you can get the academic discount, the Digilent Analog Discovery is very good value
- a microcomputer and compiler. That used to be impractical, but now is trivial; think Arduino and Eclipse-based IDEs. Use C (C++ has too many traps for the beginner), and learn to use an RTOS with multiple threads and solid real-time synchronisation design patterns
To touch on some of your other points...
When I started all I could afford was a soldering iron and a meter. My first projects cannibalised other equipment and components. Doing that is slow and can be frustrating, but it has the benefit that you have to think hard about structuring your design into bits that can be easily tested on their own. That skill will always be invaluable in a professional setting, even if the components are very different.
As for boring maths and theory, if you don't understand how/why the circuits in front of you work, how are you going to be able to predict that your novel circuits do work and will continue to work? All too often you see amateur circuits that might (or might not!) have worked
once in some limited circumstances, but which won't work for you in your circumstances.