I agree. I don't doubt that's at least a significant portion of what I'm seeing, especially with respect to noise. After all, I live within line of sight of an FM broadcast transmitter tower!
It gets weirder, though. I left the 3456A on during the day with its volts inputs shorted and the outer upper cover sitting on top of the instrument. When I got home from work the display read about -7.0 μV. I took the cover off and set it aside. A few minutes later, the display was bouncing around +/- 0.2 μV! I rested the top cover back on the DVM and the indicated offset slowly began to rise again.
With the cover on, I tried to reproduce the effect my desk lamp had, and--nothing! I knew I saw what I'd seen then, so I took the upper cover back off and bingo--instantly reproducible. I realized that I had the desk lamp pointed directly at the instrument so I tried blocking the light, and the effect was the same as if I'd turned the light off and with immediate effect. So either I'm seeing something thermal that's ridiculously sensitive, or there's something photosensitive in the input switching area.
As it happens, C104 on my instrument appears to be some kind of ceramic capacitor in a transparent glass or epoxy housing (not sure which; haven't touched it to find out). The parts list says it's just a 470 pF, 100 V DC capacitor, but for some reason HP saw fit to use a different component here instead of the identically-rated polypropylene caps specified for A20C101-103. Of course the part number and mfg code point to an HP custom part (probably a custom spec) so it's impossible to know for sure what the difference is, but it's worth at least trying a replacement.
Edit: I just tried moving my desk lamp right up to C104, and I could make the offset go as strongly positive as it went negative! I wonder if these caps were always light-sensitive, or they became more sensitive with time.
Second edit: It's not C104, but there's definitely something either directly photosensitive or something phenomenally sensitive to temperature on that board somewhere.