The answer seems to be based on how much you use the devices. For things I want to work when I need them but don't use much (like some flashlights I have stashed around the house) I spend extra for non-rechargeable lithium batteries. They won't leak and they have a long shelf life.
For things which I use often I use eneloops, which also should not leak. I use them in remotes, powered keyboards, mice, etc. But I have some devices (Russian calculators) that won't run on them, they need every bit of voltage from alkalines, or some HP Calculators for which rechargeables are not available (like the HP 41 models)...so for these devices I do use alkalines but the best I can find, and I remove them if I don't expect to use the devices for a while. My other HPs run fine on eneloops or non-rechargeable lithiums.
I've had good luck not to have many devices affected by leaks but alkalines will leak if you leave them around long enough. One Palm Pilot I have was injured by leaky Duracells, which in my experience was uncommon enough to be an unexpected, surprising disappointment.
So my general thought is to use eneloops whenever possible except for devices which get very infrequent use in which case I use the lithiums. The latter are very expensive, but not nearly as expensive as replacing precious old devices which are now unobtanium. The eneloops seem to behave well and last long enough in modern devices, and there are all kinds of chargers, some quite smart, to refresh them or at least fill them up.