The behavior sounds like the regulator isn't receiving enough supply voltage to keep running, so it conks out during startup. Then it waits for R3 to charge it back up, it kicks on, and the cycle repeats. (This is normal "hiccup" mode protection.) Too much ESR means CE2 doesn't retain enough charge during startup, and it fails to stay on.
Why does it run eventually? When components heat up, ESR falls, and CE2 operates normally again. It should "reset" on a thermal time constant i.e. it takes some seconds at least, but probably minutes, to return to the failing condition.
Tim