Nice, adding a 1K resistor across the load has got the original circuit almost functional; but still when initial power is applied the circuit is on instead of off, although subsequent switch presses work as intended.
What about changing the cap value? Incidentally, the cap I'm using is a ceramic that I bought specifically for this build and I don't have a stock of any other values - would an electrolytic (polarised) suffice, providing I have the correct value?