Here's my version:

It's elegant, but I can see a small issue with this design for the ON to OFF transition.
When the circuit is on the ON state, we can consider the capacitor discharged. The moment the button is pressed, the NPN transistor will thus switch off, which will in turn make the PMOS switch off. So far so good. The less good part is that while the button is still pressed, the capacitor is still connected to the NPN's base and will start charging through the two resistors in series on the left, up to the threshold voltage of the NPN, which will then switch on again, making the PMOS in turn switch on again!
The whole thing to consider here is the circuit formed by all 3 resistors, the capacitor AND the load! The higher the input voltage, the faster the cap will get to the threshold. Conversely, the higher the load (in terms of current), the slower. There's a whole range of conditions that can make this circuit thus turn ON again by itself while the user presses the button. With light loads, it won't take a very long press either, but a pretty typical one.
Of course one solution would be to increase the capacitance, but it may make it in turn pretty hard to get it to switch ON if you don't wait long enough before cycles.
So yes, anyway, unless I missed something (which can always happen), this circuit can't handle "long" button presses in general, and the fact that the max press time depends on the input voltage and the load can be a bit annoying.
What do you think?