Yah, you've hit all the main points.
Dropout voltage (minimum input = output + droppout) is ~0.7V higher.
Regulation is worse by the "softness" of the diode curve. It's not inaccurate by a diode drop, because of the diode compensating it. But the diode's current is ~constant (whatever the regulator GND current is), while the transistor's Vbe depends on current draw. This, plus transistor resistance, gives maybe 100mV of variation. Worse than a 7812 -- but not bad for most purposes.
No current limit, or rather it's hFE limited, which is a bad idea all the same. If you need that capability, expect to add more components to implement it. Note that it's not as simple as pulling down the base with a current sense resistor and transistor, because the 7812's current is what has to be sunk.
Nor can you quite pull it off with a LM317, which has 1.25V output, more than the Vbe drop so you can't quite completely turn it off by pulling ADJ down to GND (with a few transistors wrapping around to do that).
Tim