Tek has a probe for $$$ that has something like 50MHz bandwidth and is rated up to 20kV or so, derated with frequency. Not sure what your exact requirements are, I think that's the closest thing you'll find to a high voltage, high frequency probe.
Not sure how well DIY capacitors will perform at high frequencies (no idea what high is in this case, however). Ideally, you want the impedance and attenuation to be reasonably flat, depending on the output impedance. I would also be worried at the amount of current that fill flow at high voltage/high frequency. For example, a 10kV, 50MHz sine through 1pF will draw 3A (which is why even that Tek probe isn't rated for this). It is also likely to load the circuit.
When calibrating stuff with something like a VNA, keep in mind that components will behave differently at high voltages, many have a voltage coefficient, which is quite significant if you go from millivolts to kilovolts.