There are some CML (common mode logic), aka ECL (not actually ECL, my mistake), comparators with internal terminations which can generate 10s of picosecond edges but their outputs are not particularly clean. I was thinking of mounting a tunnel diode into a coaxial cavity like the Tektronix 284 pulse generator does.
Both of these designs point to the immediate problem; the unterminated stub length between the output device and termination limits the high frequency performance. I estimate that except for this, you should be able to get below 100 picoseconds using NPN RF transistors. In old Tektronix oscilloscope designs above 200 MHz, they got around this by running two bond wires to the base and collector of the transistor so it could be inserted between the input and output signals and their terminations but I do not know of any modern parts packaged like this.
Unless you want to go the CML comparator route mentioned above and for a printed circuit board design, I think the way to go is a coplanar waveguide implementation of the NBS (National Bureau of Standards which is now NIST) reference level pulse generator design which is very similar to the Tektronix PG506 but I have not tried it yet. These designs use a series diode in the output to disconnect the transistor from the 50 ohm termination so the most critical part of the circuit also has the simplest layout.