Is it possible to do this? I've never found anywhere it's possible, and IMO it's one of the most annoying lacking features and it gets in the way of drawing elegant neat symbols
Yes, but in another way. In the symbol editor, open the properties of a pin and disable the visibility of the pin label i.e. set visibility to pad only.
Then use the TEXT command to draw your label in layer 94 (symbols).
This way you have full control over the position and font.