The driver will try to sink the set current amount, and I can't see anything in the LCD datasheet which would suggest if it has any circuitry in there to limit the current, so you could be driving 80mA through the LED and fry it.
In theory if you use a large enough external set resistor you might be able to convince the driver that the target current is 20mA, but there's no guarantees with that driver because the datasheet doesn't actually specify what resistance is needed for smaller target currents.
I've recently used the
TLC5916 LED driver. On Digikey it's a cheaper chip (~$1.65 vs. $2.16 USD, single unit), gives you 8 output channels instead of 4, it's specified for regulating 3mA-120mA/channel with a maximum LED drive voltage of 20V, and works beautifully with 3.3V or 5V logic.
p.s. I sound like I'm a salesman for TI or something, I'm not