I'm wondering if some of the noise you guys are picking up is from secondary cosmic rays, cosmic noise in other words. Cosmic ray flux at ground level is higher than it has been due to the weakening geomagnetic field (
https://www.sciencealert.com/new-study-shows-that-earth-s-magnetic-field-is-weakening-more-rapidly-than-we-thought). This noise affects any measurement conducted over time, but obviously it depends on the sensitivity of the measurement whether or not this influences the final result.
There are algorithms for removing the effects of cosmic rays on digital images, I'm not sure about other data sets. You can get hold of curves that show the short-term fluctuations in cosmic ray flux due to solar storms (and even terrestrial storms, I was surprised to learn) and perhaps this could be correlated with periods when the noise in your measurements is higher?
If you want a real-time look at what the CRF is like in your location when you are carrying out your measurements, you could knock up a simple cloud chamber. Many designs work with dry-ice to super-cool the isopropanol vapor, but you can do the same think with butane decanted into a pyrex vessel with a covering to keep it liquid and have a copper rod as a heat exchanger connecting the liquid butane to a copper plate in the bottom of your alcohol chamber. If handling liquid butane makes you nervous I have seen designs for cloud chambers using peltier coolers, which should be ok for a small volume of vapor and as you only need to get an idea of the frequency of the rays in the same volume as your circuit's elements occupy this shouldn't be a problem (the noise affecting a whole instrument rather than just the voltage ref might be a bit trickier... especially with something like a Fluke 5440