I'm not very familiar with the LT3080, but a peek at it's datasheet mentions a minimum load of 0.5mA. (Reference Buffer, Page 19). I assume you added the LM334 for this, but that IC probably stops working properly when it's voltage gets too low.
Same is also true for the LM358. It also has some trouble with low output voltages. You could add a resistor between it's output and GND (or lower the impedance of R4, R5, R6) to pull it to GND "harder", but it quickly becomes a comprise between the ability to lower the minimum output voltage, and having a reasonable dissipation at the maximum output voltage.
Designing a power supply (for low output voltages) is much easier if you have an extra negative voltage available.
I also wonder why you added R3. Together with R1 it forms a voltage divider, which means that the opamp has to amplify more, and also amplifies it's input noise. (Which you partially filter out with R7 & C4).
The reason I have never used the LT1080 myself is that I do not find it a very useful IC, and it's too expensive for what it does. If you replace it with a simple NPN BJT in this circuit, (And use feedback direct from the output) then the LM358 never has to control it's output below 600mV.
You loose the current limit of the LT3080, but any decent power supply has an adjustable current limit anyway.
During power-on the LM358 is also not guaranteed to work properly. You need some kind of circuitry that prevents any spurious voltage on the output of the LT3080 during power-up of this power supply.