I took a step back for a few days but decided to have another look it today and I think I might finally be getting somewhere. When I previously removed the reference and replaced it with a transistor and 5.6V zener,, I found that the output stayed high at around 9.4V making it seem that the situation had been made worse. Replacing the reference seemed to improve although not solve the problem. However, looking back at my notes, rather embarrassingly I discovered that I had fitted the zener incorrectly. The orientation of the transistor was fine, but when soldering on the zener I had forgotten to take into account the fact that when fitting it to the underside, the connections on the flipped board would be in reverse. I had re-checked everything at the time, but somehow still missed this. Therefore the first thing I did was to remove the reference again and replace it with the same transistor and 5.6V zener diode that had been used the first time around, but making sure that the cathode of the zener diode was soldered to the correct pad.
On powering up, the rail started at around 7V, rose quickly to around 9V and then over 5-6 seconds sunk to 6.53V. I was initially disappointed, but then realised that this was different from previous attempts where the excessive voltage was present immediately at power on. A settled level of 6.53V was fine because this is approximately what the SPICE model had predicted for a 5.6V zener diode. I left it powered down for around 30 minutes and then tested again. The rail immediately came up at 6.53V. I have since left it powered down for over a couple of hours and have just tested it again and this time also, on the instrument being powered on, the rail started at 6.53V and stayed put.
Query: might the effect observed on first power up after fitting the alternative parts be down to residual charge in the smoothing capacitor? The zener initially clamps the rail voltage, but then becomes "saturated" by the residual charge held in the capacitor until it discharges to an appropriate level, whereupon it settles at the expected voltage?
Whatever the case, so far it hasn't done that on subsequent power-ups but has started at exactly the expected voltage. I will test it again after a couple more hours and again in the morning just to be sure, but if the rail still immediately comes up at 6.53V, then the fault must evidently be inside the reference. Incidentally, in the meantime, I have been experimenting with the SPICE model and found that severing the connection between the zener and the emitter on the transistor, or at the transistor base, results in an elevated output voltage of around the level being experienced. Could it be that one of the bonds inside the reference is being affected by the reference warming up? This did have the "feel" of a thermal problem, although one that has been rather elusive and inconsistent with whatever else I was doing on the board in order to trace the problem.
UPDATE: Noooooo! Its back to the same behaviour again this morning. Really thought I was on to something this time!