A square wave must have harmonic frequency content. Let's say up to the 5th harmonic: 25 MHz for a 5 MHz wave.
That is roughly a 0.35/5MHz = 14 ns rise time.
14 ns from 0 to 5 kV is a 5kV/14ns = 357 GV/s slew rate.
Now suppose you have 1 pF of parasitic capacitance. This will conduct I = C dv/dt = (1pF)(357GV/s) = 357 mA.
That is about 1785 W instantaneous power, though the average power is a bit tricky to compute.
In short, you don't even want to have a 5 MHz 5 kV square wave, as it will be nearly impossible to contain. Forget actually generating one.