A low ESR is often wanted, but not Always. And every cap has its pros and cons. Ceramics have low ESR but suffer from piezo effect. Some types of caps have huge voltage and/or tempco's others not. Mica's are not great at very low frequency (around 10 Hz) and MKT's are not great at very high frequency. Two caps parallel lowers ESR but increases selfinductance
Tantalum has low ESR and can last for ever, but are easy killed by over-voltage, some caps are low leak, others are can withstand pulses better.
Close to the IC has to do with the circuit layout. The trace between cap and IC, together with a possible input capacitance of the IC can make a nice LPF, sometimes wanted other times not. There are pcb's where the selfinductance of vias and traces can cause a lot of problems. The best place is making it part of the whole design.
Download the Agilent impedance handbook and there also is an appnote about ESL in caps, via's and traces.
A while back I had to filter some powerrails in an RF generator. I went a bit over the top with placing caps, ferrite, CLC filters and RFC's on strategic places. Measured impedances etc and it had to work great. It did work great in the test with almost no load. But while testing with a dynamic load one rail started oscillating. After I found the cause a smal series resistor to increase ESR for a cap solved the problem. Not very strance, take some capacitance, add a sniff inductance, add some feedback and you have an oscillator.