I'm still not sure what to conclude, but it seems now, after a few days running idle, being also off for a couple of days, and then being turned back on and let warm up for a few hours the offset seems to have stabilized very nearly dead on zero (all three). The DC SG zero is actually almost dead on .0000(-3)V, with a bit of fluctuation between the + and the - polarity of that (and when straying slightly from it, it'd occasionally go to up to -.0002(-3)V, which I have no issues with).
The other two "zeroes" are less than 5-7 counts off (.000005-.000007V).
If this stays this way, I think I can go ahead and readjust the ADC ladders and the rest of the adjustments. Not sure what I should be doing if after that I get an offset in the uV range again.