32-bit address space is not an artificial limit. Extending that with PAE is a hack job akin to HIMEM.SYS from DOS days. It's a stupid solution and should never be used. It's like building a shed on the back of your house. Every time you fill your house up, you have to go back and forth to the shed to reorganise stuff. Answer is buy a bigger house.
LME incapable or buggy as fuck drivers which I've encountered: FC cards, PCI serial cards, Matrox graphics cards, virtualization drivers (vmware workstation).
Also note that drivers which aren't compiled with LME actually have to copy memory twice in and out of the address space of the CPU so you get shitty performance.
Also even if you have PAE enabled, if hardware takes memory away, then that's taking it out of the 32-bit address space of the CPU, not from the total PAE allocated space.
Just use a 64-bit OS. There's no reason not to, unless you have some weird driver which is 32-bit only or something at which point you should start thinking about replacing that hardware.