I've gotten the boards back and soldered everything.
I had to make a few bodges to get everything working. I had issues with oscillating, so I removed the filters and I just passed everything straight to the FM input. I also disconnected the main tuning input to try to isolate the problem. Now I just turn the front panel knobs until it's close enough for the PLL to lock on.
So far I have been able to get it to phase lock correctly. However, there is a lot of phase noise. I have tried the following things:
1. Checking noise levels around the power supply. I used my HP voltmeter which has bandwidth up to 250kHz. The 3.3V has no measurable noise, the 5V has maybe 30uV RMS, the 20V has maybe 200uV noise and the -10V has 100uV noise. The op amps should have high enough PSRR to reject any noise on these rails though.
2. Running the VCP of the PLL IC off the 3.3V instead of 5V supply. The 3.3V has much lower noise because it goes through a good LDO.
3. Changing the loop filter to one with higher phase margin. The values shown on the schematic attached are the old values which have 40 degrees phase margin, but changing the phase margin to 70 degrees made no difference. The loop BW is still 30 kHz.
4. Measuring the residual AM on the output to make sure I'm not seeing AM modulation. I hooked up a detector to the output and I read 0.12V DC. I got an AC reading in the microvolts range.
I'm wondering if the phase noise could be due to the crystal I'm using, which is a cheapo 100MHz crystal from YXC. Here is the datasheet:
https://mm.digikey.com/Volume0/opasdata/d220001/medias/docus/5344/YSO110TR.pdfI also noticed that the S11 of the input sucked (it was >-1dB at 2GHz). Despite this, the divider inside the chip works. Using one of the multiplexed pins, I "broke out" the internal N divider and I measured it with an oscilloscope. It is at the correct frequency.
I disabled Fractional N in software, and I am using an integer N (42), along with a 100 MHz crystal to generate 4.2 GHz.