IIRC its possible to use the Z80's Refresh signal + a moderate amount of LSI and MSI logic to generate video. Obviously you need some external counters and dividers to generate the dot clock and Hsync interrupt and a PISO to serialize the video. Z80 refresh normally steps through a 7 bit address range accessing one location in each M1 cycle, but some relatively simple extra logic circuitry can extend that to 8 bits (you need to switch between normal A7 when /RFSH is high and when its low, A7 from a flipflop toggled whenever A6 goes from high to low between consecutive refreshes). The high byte of the refresh address comes from the I register, so can be handled in software in your Hsync ISR. The Z80 instruction rate is variable due to their different lengths, but for a 4MHz Z80, it should average 30 accesses in the 52us of the visible portion of a PAL scanline, which if you let the FIFO preload a bit in during Hsync is enough for 256px per line mono video. Add a moderate depth FIFO, and in your Hsync interrupt, reset the I and R registers (and your external A7 flipflop) to the correct address for the start of the next scanline and also clear the FIFO to discard any overrun. Underruns are a little harder to handle, but if you arrange for your FIFO to raise a NMI before it runs out, worst case you'll get a /RFSH cycle 25 clock cycles later, then another 4 cycles after that as execution reaches the NMI vector, then the NMI routine can execute a bunch of 4 cycle NOPs to rapidly re-stuff the FIFO enough to see out the scanline before returning.
That will give you a mostly full speed Z80 with interleaved video memory access. The Z80 will only be slowed down by your FIFO reload NMI ISR occasionally if there's a run of long instructions in your code.