Welcome to the ranks of the microwave-philes!
If you're just getting started, check out some of the ARRL publications on UHF/VHF/Microwave experiments and construction. RSGB also has some very good material. Even the older stuff is informative.
For what it is worth, I've build a lot of 10GHz equipment with not much more than a power meter. But I'll say that a signal generator (not a synthesizer, just a generator) was rather handy. Even at that, some of the weak signal sources (an oscillator at 1152 MHz run into an amplifier that is well beyond its linear region) work just fine to set up receivers. Depending on where you live, there may be beacons on 10GHz and lower bands.
For the 10GHz antennas (mostly offset-fed dishes) I've used tried and true feed designs. Most of the tuning has little to do with the impedance match, and much to do with the position of the feed relative to the dish surface. (Yes, we learned in physics that the feed center has to be at the focus. Alas, they never told you precisely where the feed center would be.) For alignment, and general RX performance measurement, you need two things: a big honking source of broadband RF energy like a thermonuclear event, and a sensitive power sensor. For the former, I use the Sun. It is available for much of the day, though not every day. For the latter, I use the panadapter display on my IF receiver (it is a very fancy SDR, but you could use one of the inexpensive dongles.)
For lower band antennas, a power meter and a directional coupler -- two if you're lucky enough to find a good deal -- is really all you need. Yes, you don't get the phase with a power meter, but you get the magnitude. When you're standing next to a 15 foot antenna tweaking a loop and finding the right tongue angle, one dimension is about all you can optimize over.
If you really want to measure magnitude and phase, the least expensive technique I know is with a slotted line, a probe, a signal generator that can modulate a 1kHz tone on the carrier, and an ancient HP 415 SWR meter. There ought to be something on the web about how one of these works. I've done that once or twice, but found little value in it.
Far and away the biggest bang for the buck in my shop has been my HP432A power meter. I'm partial to the old fashioned kind with a needle. When you're tweaking, you look for the needle to move in the good direction. It is much more natural to react to a needle moving than to recognize that 2.38 is greater than 1.38. (And the needle puts in perspective just how insignificant 1dB really is, in the grand scheme of things.) Be aware though that you can get the meter itself for cheap on Ebay or other places. It is the measurement head, and -- believe it or not -- the cable between the measurement head and the meter that are rare and often expensive. Don't buy a meter body without the cable and head. My goto measurement head is an HP 478 that goes from about DC (100 MHz) to just shy of daylight (10GHz). These are fragile -- I bought one of mine from a "trusted source." The second was a wedding present from the same "trusted source" as we had done a fair (ahem) bit of business by then. My wife was touched. The meter tops out at about 10dBm input, so you need a stack of attenuators and an occasional directional coupler to make it useful.
By mass, my shop may be dominated by adapters to go from one RF connector family to another. All the good stuff that I connect up is either SMA or N. But every once in a while there's some joker with a BNC or even a UHF connector. (Queue the chorus of replies of the flavor "yeah, adapters, I've got a BillumQuick 5832 Microwave Wobulator with APC-7 connections on one side and a three pronged Russian triax connector on the other. Finally found the adapter at Pasternack for a little less than the cost of a Ferrari.")
This setup won't rival the average dumpster at NIST. And, notice, that I haven't mentioned a spectrum analyzer yet.
I have a spectrum analyzer. It is amazing how little I use it. It gets powered up every now and then, but it is creaky, has grown spurs, and does everything but leak oil. I do, however, have some bodacious broad-band RX, signal generators that, between them, cover 10MHz to 13 GHz, a power meter, and the ability to make a filter or two when required.
Enjoy building stuff. Don't let the absence of a piece of fancy test gear get in the way.
keep your antenna on the ice.