Actually, thinking about it some more... all the devices you've mentioned are geared up for a 68xx / 65xx type of bus, and rely on shared memory. They might be a pain in the arse to rig up to a Z80 (though possible, you'll need to do some fiddling).
What might be the best option is to get hold of a TMS9918/29 instead. The Sega Master System used a similar and mostly-compatible VDC with a Z80 CPU, and the 9918 & variants use their own separate DRAM-based video memory. It handles all the timings and addressing for you, so no need to worry about the black arts of DRAM.
IIRC, you write to it just like an I/O port, and would need some sort of interrupt setup so that the video memory is only updated during the vertical blanking interval.
Search for a schematic of the master system and you'll get a feel for how it all fits together.