1) Start off with one of those 0,1 inch breadboards, aim for something reasonably sized because your projects may get larger later.
2) Purchase a bulk pack of resistors in all of the E24 values and a bulk pack of capacitors in all values from (say) 1pF to 100uF. Ebay is one source for these but you may find them on Amazon or (even better) a local components shop.
3) Also buy 50 x 2N3904, 50 x 2N3906, 10 x 555 Timer IC and 10 x NE5532 Op amps. Maybe add 50 x 1N4001 diodes as well, any other semiconductors can be purchased as needed, for example LEDs.
This should give you a great start in electronics but there is some other stuff that you can do to help things along:
4) Keep an eye out for stuff thrown out as trash. An old TV set can be a source for some useful components but an old radio will be better.
5) If you have a Fablab or a Maker Space in your area then head along and say hello, there's always one or two electronics geeks hanging out.
6) Try your local Ham radio club, hopefully they will make you welcome.
If you want some links to websites that have lots of circuit ideas then let me know.