An FPGA won't have any problems driving these loads (6.6mA @ 3.3V), and I've used it with 100+ MHz pixel clocks with no issues.
I still feel a bit reluctant to expose my sensitive FPGA pins to the world albeit through a resistor network.
Just add dual schottky diodes right at the VGA plug to GND and 3.3v to clamp the signals from the VGA plug.
Unity gain rail-rail 3.3v high speed analog buffer amps will also work. Use 2x 3 channel ones, 1 for the RGB and the second for the syncs.
When using the unity gain amps, don't forget to add a 150-470 ohm load at the input and a 75 ohm series resistor at the output.
Even a simple high speed PNP emitter follower, like a MPSH81, with a 75 ohm pullup on the emitter + 75 ohm series resistor to the vga plug would also work.