Applications of Sparse L1 Pursuits to Precision Reference

[rhb]

The  properties of sparse L1 pursuits were initially presented in a flurry of papers by David Donoho and his former student Emmanuel Candes.  Candes had done an experiment in which he attempted to fit random signals with a combination of Fourier series and impulse spikes.  The results were far better than expected and led both to an intense effort spanning most of 2004.

The best summary of the importance of their work is stated in the introduction to this paper:

https://statweb.stanford.edu/~donoho/Reports/2004/l1l0EquivCorrected.pdf

Unless you enjoy mathematics a lot, I suggest just reading the introduction.  Why it works is not as important as that it does work and that software for applying the technique is readily available.

What Donoho proves in the paper cited above is that a least summed absolute error (L1) solution to an ordinary system of linear equations Ax=y is exactly equivalent to the optimal (L0) solution which is NP-hard and can only be obtained by exhaustive search.  Previously, the only known solution to such problems was to try every possible combination of the columns of A.  For significant problems this is impossible.  On the fastest computer available it would take until all the stars in the universe burned out.

This has profound implications for a great many applications.  In the context of metrology, it permits solving problems which account for all the known and characterized system errors.  In the case of errors due to quantum effects which can only be described statistically, it provides tight bounds on the irreducible error.

So if we parameterize the system output  of a voltage reference and the associated buffer amplifier with an expression such as:

expected_value = vref_initial_value + ref_aging + vref_1/f_noise + vref_temp + amp_initial_offset + amp_aging + amp_temp + r1_initial + r1_aging + r1_temp + r2_initial + r2_aging + r2_temp + r3_initial + r3_aging +r3_temp

Given reasonable approximation of the values in the summation as arbitrary functions of measured parameters and unknown coefficients, we can compute the functions for a large number  of possible coefficients.over a wide range.  A problem with 10,000 choices for each term is solvable in a matter of a few minutes by means of linear programming using the simplex method.  Faster algorithms based on the properties of regular polytopes in N dimensional space exist.  But the simplex method, invented by Dantzig  to solve optimization problems in operations research does an excellent job and high quality FOSS  software is readily available.

If one computes 10,000 possible instances for each of the terns above as functions of measured values such as temperature, age, etc, there are 10,000**16 possible combinations.  An L0 solution is computationally intractable.  It can't be done.  But an L1 solution, if it exists, has been proven to be identical if the columns of A are uncorrelated in all combinations and the x vector in x of Ax-y are sparse, that is, most of the coefficients of x are zero.

To put this in concrete terms, suppose we know from testing that the value of a resistor is f(a,b,c)  where:
a=initial value
b= change due to ambient temperature
c=change due to current flowing through the resistor

but that we do not know  a, b or c.  All we know is that the change due to ambient temperature  is some function g(b) and the change due to heating by the current is h(c).  If we know the form of h(c) as say a polynomial h(c) = k0 + k1*c + k2*((c-d)**2) from measuring a number of resistors where k0, k1 and k2  are all vary with each device, then we can evaluate a large number of values of h(c) for a range of values of k0, k1 and k2. All of these are added to the A matrix as possible answers.  The "dictionary" in the terms of sparse pursuits.

So what one does in practice is create models of the functional forms of the various terms.  One then builds a massive dictionary of all the possible values for each term as a function of the known values.  Unknown values are simply varied over a range of expected values and the expression evaluated.  In the example cited, with 10,000 instances for each term the A matrix would consist of 160,000 columns and as many rows as one had measurements.  When i was in grad school 30 years ago and are computers consisted of an 11/780 and a MicroVAX II solving such a problem was not possible even if we had known how.  Now it's no harder than computing a long FFT on a desktop machine.

[texaspyro]

That made my noggin throb...   

[TiN]

It's all fun and dandy, but do we really need third thread on essentially same topic? 

[ramon]

Yes, we do need a third thread! I really want to know if we can predict noise with that sparse L1 evil thing.

[zhtoor]

Yes, we do need a third thread! I really want to know if we can predict noise with that sparse L1 evil thing.

it is not about predicting noise, it is about predicting the drift pattern based on some sophisticated model and adaptive predictors (like kalman filters) if at all.

regards.

-zia

[IconicPCB]

... and the rope gets longer...

[Conrad Hoffman]

There are three kinds of people, those that understand advanced math, and those that don't.
 

[Theboel]

There are three kinds of people, those that understand advanced math, and those that don't.
 

and the third are ?

[ramon]

Zia, the words were not randomly chosen. Why do you make such difference between drift and noise?

[zhtoor]

Zia, the words were not randomly chosen. Why do you make such difference between drift and noise?

look at LTZ1000 datasheet, on the first page, look at "FEATURES", and i quote:-

1.2uVP-P Noise
2uV/rtkHr Long-Term Stability
Very Low Hysteresis
0.05ppm/°C Drift
Temperature Stabilized
400°C/W Thermal Resistance for LTZ1000A Reduces Insulation Requirements
Specified for –55°C to 125°C Temperature Range
Offered in TO-99 package

best regards.

-zia

[rhb]

Sparse L1 pursuits are far more general than the particular application of the OSHW voltage reference which is an inverse problem.  It is the basis of the solution that won the Netflix prize for best predictor of other movies people would like.  In mathematical jargon that is called "matrix completion".  Other applications in include error correction, machine learning and compressive sensing.

I started this thread specifically to address some ideas for improving high precision measurements. I think it unhelpful to clutter the OSHW Vref thread with details which are mostly germane to sub-ppm cases.  My intent with the OSHW project is a cost-performance design effort.  This thread is is best possible performance. National lab level work.

I also wanted to separate out the mathematical details so I didn't find myself writing the same thing multiple times to explain a point.

Consider measurement of a standard resistor in an oil bath with precise control of the temperature and  measurement current using a state of the art voltmeter, the details of which will not be treated except in the abstract.  Initially I shall presume that the meter is perfect.  Such things do not exist, but I want to keep the initial discussion as simple as possible.  The complications  of real meters will be dealt with later.

No matter how precisely we control the oil bath, current flowing through the resistor will produce self heating and unavoidable thermal gradients in the measurement path.  The current under consideration will be limited such that it does not cause permanent damage to the resistor.

When the measurement is started, the resistance will change as the current heats the device.  This is a simple curve which asymptoticaly approaches a steady state value.  However, we should have to wait forever to reach that value. The mathematical form is an instance of Fourier's heat equation, so the asymptotic behavior is well studied.  Measurements during the start of the run will allow very precisely determining the value at any instant in the course of the measurement run.

For the purpose of this discussion, I shall stipulate a wirewound resistor.  With that construction the resistor also has an inductance and an interwinding capacitance.  That might seem irrelevant to a DC measurement, but quantum effects come into play.  In particular, the current source will have a number of quantum level noise components, 1/f being the most troublesome.  The resistor itself is also a source of noise.  As the latter noise source is broadband, the parasitic inductance and capacitance of the resistor form frequency dependent filters that will alter the value of the noise at certain frequencies.  These effects will be ignored for now.

To make a precise measurement of the resistance using a precision current source we need to account for a number of small, but vexing problems.  We have thermal noise,  i/f noise and more.  At present I know about the form of the solution but not the equations of the problem. I shall limit myself here to just the thermal noise and the 1/f noise.  The first two layers of the onion we wish to peel apart.

If we take a long series of measurements which start before power is applied to the resistor we will observe the effect of heating of the resistor and we will observe all the various quantum level noise effects produced by the physical devices in the circuit,

If we Fourier transform this series of measurements with our ideal meter we shall see the 1/f noise of the current source, the thermal noise of the resistor which is at a slightly higher temperature than the oil bath as it is a heat source and  the Fourier components of the resistance change induced by the current flowing through the resistor.  The latter will change  over time as the resistor heats up, but I shall ignore that complication for now.  It can be dealt with at the cost of making the system of equations larger.

At present I shall assume that the 1/f noise is the least well determined error term and that the self heating is adequately described by the heat equation so that we need only determine the asymptotic value and the time constant.  The equation is precisely known and we can calculate the value during the course of the measurement for a large number of possible values at infinite time and a range of time constants.

The thermal noise of the resistor is also known to a degree, but we do not know the exact temperature of the resistor, just the temperature of the oil bath. Again, we can compute the power spectral density of the thermal noise for a range of temperatures covering the expected values based upon the temperature of the oil bath.

This leaves the 1/f noise of the current source.  This is the most difficult term in our problem.  Obviously the noise does not become infinite after an infinite period of time.  So clearly, the correct functional form is more complex than 1/(f**a), though that may suffice for measurements over practical time periods.  A variety of more complex forms have been proposed.  Once again we can compute a large number of estimates of the power spectrum density for a range of coefficients and estimation functions.

With the exception of the self heating error, the other components are essentially random. We can make some general statements about the expected values of the power spectrum such as the mean and standard deviation but no more.

So our mathematical problem consists of D(f), the Fourier transform of the measurements, and the dictionary of error term models for the resistor and the current source.

D(f) = R(0) + F(f) + S(f) + G(f)

Where:

R(0) is the DC component of the transform

F(f) is the 1/f noise

S(f) is the transform of the self heating

G(f) is the approximately Gaussian thermal noise

Thus we have Ax=y with a rather large A matrix and an x vector which should contain 4 non-zero elements which give us the DC resistance, the coefficients of the 1/f noise, the actual device temperature and the time constant and resistance change of the resistor due to self heating.  The next layer of the onion is to account for the voltmeter at a similar level of detail by adding the appropriate equations to the system.  To measure the drift the problem outlined here would need to be be modified to to use R(f) instead of R(0).  As a practical matter solving for R(0) once per measurement run and then using those results to solve for the drift equation is more tractable to set up and solve.

To the best of my knowledge no one has attempted anything comparable.  Should anyone know of work at this level of refinement I should be most grateful for references and contact information.

[MisterDiodes]

RHB:

So lets get back to a practical application of "Chasing a few little ones" - And how it works on Reg's Cheap Calibrator for example, or even on a good Vref.

Without dropping author names, your credential history or equations, walk me through a one paragraph explanation of why you think this might work on a system that also includes an unknown chaotic history based real worlld situations:

Let's say you find a Vref out of the blue that you've supplied a couple years prior and maybe at some point a few of it's "L1's have been pursued".  In other words you think you might be able to predict it's actual Absolute Value of Voltage to some level of uncertainty at some point in the future.

BUT - There is no known history of the device in the few years it's been in the field.  You have no idea the time-integral effect of load, humidity, temperature effects, voltage bias points, etc, etc, etc.  Was the device shipped across the Arctic freezing on an airplane or did it sit in a shipping container for months, or was it in constant use the whole time in a lab.  All of that is unknown.

A really good one that throws the college interns claiming to be able to predict a future value is having them look at the RATE of CHANGE of temperature on a Vref or resistors, and you can also look at the Jerk effects as well (rate of rate of change).  See if that's predictable and how that affects long term drift.  That's an effect that we look at in real world situations too, all of which have long-lasting effects on variables that you could have sworn you had a handle on.

The point is: you are holding in your hands a device on which you have NO idea of it's past use history or environmental exposure:  How does an L1 pursuit technique predict it's current value, especially when you have absolutely no clue about it's past history?

[zhtoor]

RHB:

So lets get back to a practical application of "Chasing a few little ones" - And how it works on Reg's Cheap Calibrator for example, or even on a good Vref.

Without dropping author names, your credential history or equations, walk me through a one paragraph explanation of why you think this might work on a system that also includes an unknown chaotic history based real worlld situations:

Let's say you find a Vref out of the blue that you've supplied a couple years prior and maybe at some point a few of it's "L1's have been pursued".  In other words you think you might be able to predict it's actual Absolute Value of Voltage to some level of uncertainty at some point in the future.

BUT - There is no known history of the device in the few years it's been in the field.  You have no idea the time-integral effect of load, humidity, temperature effects, voltage bias points, etc, etc, etc.  Was the device shipped across the Arctic freezing on an airplane or did it sit in a shipping container for months, or was it in constant use the whole time in a lab.  All of that is unknown.

A really good one that throws the college interns claiming to be able to predict a future value is having them look at the RATE of CHANGE of temperature on a Vref or resistors, and you can also look at the Jerk effects as well (rate of rate of change).  See if that's predictable and how that affects long term drift.  That's an effect that we look at in real world situations too, all of which have long-lasting effects on variables that you could have sworn you had a handle on.

The point is: you are holding in your hands a device on which you have NO idea of it's past use history or environmental exposure:  How does an L1 pursuit technique predict it's current value, especially when you have absolutely no clue about it's past history?

lets simplify things a little bit to make this discussion more fruitful:-

assume:-

1. a JJA is available right next to your setup. (for periodic / aperiodic transfers)
2. you are running a setup which includes some kind of a predictor (L1 persuits, kalman filters, sensor fusion, neural networks etc.)
3. now you have a unit under test (UUT) which is being monitored and its predictor updated at pre-deteremined times.
4. a particular UUT is characterized using this method and dominant environmental parameters established from model analysis.

and then:-

build a UUT with the dominant parameter logging / recording built-in to enable UUT transport / transfers.

best regards.

-zia

[rhb]

If one has no knowledge about past device history, the only thing one can do is ascertain the effects of noise on the measurement in a more sophisticated fashion than merely averaging values over a number of power line cycles.  Even the most modest consideration should make that obvious.

This is NOT the OSHW Vref project thread.  It is about developing  system models for precision references and using those to improve the accuracy of the measurements.  My example was intended to illustrate the concept, not be a comprehensive description of the physics of  a resistor or the mechanics of solving the problem.   To the best of my knowledge, this is a national lab level subject which has had little or no investigation. 

As you seem to resent mathematical analysis, perhaps you should sit this one out.  I believe the jargon is, "This is a league game".  This thread is about the application of recent developments in applied mathematics to state of the art problems in metrology. 

[MisterDiodes]

I guess I'm still confused when you started with a $50 Vref with 4ppm drift using cheap resistors and "...Being...ignorant", and then the discussion moved to describe a $150 calibrator for 5.5 digit DMMs...and now you're doing an NIST project.  OK, I'm just trying to keep up. I thought the threads were somehow related.

Don't get me wrong - I'm always interested in a REALISTIC and ROBUST mathematical approach to make GOOD measures and equipment BETTER.  We use those techniques all the time!.   The reason we like to have fun with the summer college interns is hopefully they go back to university with eyes opened a bit - again we try to get them to put down the books for a sec and observe the real world.  Because half the time their profs haven't got a flippin' clue of how or why or when to apply the math.  It's bad enough we get EE kids that have no idea of which end of the soldering iron to hold (They can simulate it though!)...but that's a different story. 

OK, OK - I promise my mind is open, but I'll just pull up a chair and shut up.

Sell it to me!

[rhb]

I should like to pose a few questions.

The procedure I intend to follow  is to develop a system model, make some measurements, setup Ax=y, solve it and then examine how well the model matches future measurements.

I'd like to do this with existing data to start.  TiN has made a lot of data available.  I've not yet looked at more than a small fraction of it.  So I should like him to suggest the best choice.

I want to address 1/f noise and thermal noise in the first iteration.  The data requirements for this are measurement runs of moderate length with temperature data.  More environmental data may be useful, but this is the minimum.  It does not appear that it matters whether the measurements are resistors or voltage references.

For resistors we have the 1/f noise of the ADC voltage reference, the  voltage reference and ADC input buffer amps and the current source and its associated circuitry.  Are there other components which generate 1/f noise?  We  also have thermal noise from a number of resistors besides the DUT.

For voltage references the noise sources are similar except that we have 1/f noise from the DUT rather than the current source.

Because I propose to address the 1/f noise in the frequency domain, long measurement runs are important. My present candidate would be the cyrogenic test in February just looking at the data when the device was normalized to the liquid, but I shall consider any other data that meets the requirements for setting up the problem.

Whatever problem is chosen, I need an equivalent circuit model which includes all the noise sources.  AS I am not familiar with the details of the instruments I'd much appreciate of someone who is would define the equivalent circuit model,  I shall have to write several programs to generate the GMPL input to the glpsol program fin GLPK and to do that I have to work out how to properly linearize the problem.  So it will take a while to do this.  The input problem files will be many megabytes, so writing them by hand is not a practical undertaking.

I also need  references to the professional and OEM literature.  Typically for a serious project I will collect and read 50-100 papers.  I usually try to find a few good recent papers and then go back through the references in those.  I shall be spending sometime at the nearest university library on the 10th, so if anyone can suggest a few good starting points related to 1/f noise in voltage references, zener diodes and op amps that would be very helpful.

I'd like to keep a clean separation between this and the OSHW project.  This is a science project.  The OSHW is an engineering project.  If this effort is successful then it can be applied to the OSHW project during the development of the UI and calibration component of that device.

[MisterDiodes]

You can look at a PWW resistor which has virtually no 1/f noise to begin with...That might narrow down the variables you're looking at on a first pass, and allow you to concentrate on other effects.

EDIT:  Use the same approach with the amplifier for your first tests:  Use a chopper amp to eliminate 1/f noise contributed by the amp.

This might be something to look at in your L1 Pursuit / OR1MP experiments as well, from an overall efficiency point of view:  At what point does spending slightly more money (overall) on good components leave a smaller carbon footprint than using less stable components that require a lot of CPU cycles (= Energy = Cost) forever trying to compensate for more noise and drift?

[rhb]

Mathematically, I don't think the complexity matters.  Mostly I want to avoid spending a huge amount of time researching the noise characteristics of numerous  devices.  However, I do want to be complete to first order.  The most important aspect is to use existing measurements for the first few trials.  So error terms for which the associated variables were not measured will be left out.

Not sure what you mean by "OR1MP".  Could you please explain?  One only need  solve the inverse problem when a new calibration is done.  Otherwise it's a very simple evaluation of Vdc = F(temperature, age, etc).  Something very trivial for an MSP430 to do for a few picowatts.  The RTC and display will consume vastly more power.

There is no way to compensate for the noise.  It's important to include it though.  If this effort succeeds it  will dominate the error bars.  The point of including it in the system model is to remove the effects of noise from the aging function estimate and the estimate of the true value.

I found a paper on resistor noise which looks to be fairly complete.  I'd be grateful for a review with particular attention to omissions or errors in the equations as well as links to other papers.

https://dcc.ligo.org/public/0002/T0900200/001/current_noise.pdf

This thread is really about improving the calibration process for electrical measurements and references, particularly transfer standards. I'll leave the national lab staff to apply the techniques to primary references if the method succeeds with transfer standards.  If it meets my expectations for aging and temperature corrections I'll take a crack  at the hysteresis effects which plague traveling references. That may not be solvable beyond a very  limited degree.

For the first trials I'm looking for  a nice smooth, repeatable temperature curve separated from noise curves which are uncorrelated from run to run.

Errors in the DMM will be largely uncorrelated with the reference being measured.  So by reading two or more references during a run the low frequency portion of the DMM errors can be separated and corrected.

The mathematics of doing this are just basic algebra.  The mathematics of the solution are ugly, but there's no real reason to go into that in detail unless edge cases arise.

[MisterDiodes]

OR1MP = Orthogonal Rank One Matrix Pursuit (or just Rank One Matrix Pursuit) which is not the same as your Sparse L1 Pursuit I know, but produces -very- similar results, at least on the places where I've seen it used.  As you would know.  That's what I thought of based on your initial description, and I got that stuck in my head.

That paper you reference above even mentions on Page 8 that PWW and BMF resistor have little to no excess (1/f) noise.  I'm not sure but I don't think they have many PWW resistors represented on their graphs, but double check.  A few of the model numbers I'm not familiar with.  One of the mistakes in that experiment of course is the use of an aluminum shield box, which does just about nothing for low freq EMI - which will find it's way into the resistor test.

The man who wrote the book on PWW is here on the forum, Edwin Pettis, and I suggest you check his past articles - and the fact that PWW are devoid of 1/f this is also something that's been known for many decades in the older literature (back to the 50's and before), a lot of which is not online.

The same mechanisms that produce VC and 1/f are present in all other resistors because (very simplified) a PWW uses a round conductor, and when it's properly annealed and formed that offers very few areas where electron traps can form, and the atomic bonds are all of very "strong" type.  Any time you try to use a planar resistive element you'll run into many 90° corners of the conductive structure, all of which can form electron traps which add to -chaotic- electron flow (noise) - and flat, non-smooth planes are areas where tend to find a lot of weak atomic bonds (allows atoms to swivel around and align in an E-field), tending to produce VC effects.  In a very simplified nutshell.  We run into these problems all the time on film / diffused resistors built onto a crystal lattice such as GaAs, Si, Sapphire, etc.

But Edwin is the expert in this department, and he can add more.

 

[zhtoor]

hello,

please find attached test bench #1:-

1. there are N voltage references operating in tandem.
2. the N voltage references are connected to switch groups SA(1..N) and SB(1..N).
3. any of VREF(1..N) can be selected under automated control of SA(1..N) to produce average of the selected refs on line VA.
4. any of VREF(1..N) can be selected under automated control of SB(1..N) to produce average of the selected refs on line VB.
5. line VA and VB is measured differentially and analyzed.
6. it is assumed in the start that switches SA(1..N), SB(1..N) and resistors RA(1..N), RB(1..N) are noise/thermal emf free.
7. the maximum difference between VREF(1..N) is under K ppm.

to solve:-
1. which subset of the VREF(1..N) is an outlier and by what margin?
2. which subset(s) of the VREF(1..N) form a group in such a way that their combined drift is a minimum?
3. what are the individual drift patterns vs. the most stable group?

best regards.

-zia

[rhb]

@MisterDiodes  I'd never heard of OR1MP.  I'll need to look into it.  There were a number of sparse L1 pursuits in use by a wide variety of names before the work of Candes and Donoho. 

@zhtoor  Geez!  Did you need to bring such a *big* baseball bat? ;-) 

I figured out how to remove the measurement system reference error.  That's correlated between the measurements  You have posed a seriously difficult problem.  Do you know of a solution or is this an open problem?

I've been contemplating this for N = 3 without success. I haven't been able to determine even whether it can be solved.

I'll work on it some more.

[rhb]

Professor zhtoor :-)

If we measure the output of N references over a sufficiently long period of time and crosscorrelate the N references with the N-1 other references, the random noise will have zero correlation.  Therefore the references with the largest crosscorrelation coefficient will have the most similar drift.  This result is independent of the thermal noise of the switches and resistors.  So it applies even if those are non-zero.

The selection of the best set of references may be done by sorting the references by crosscorrelation coefficient and selecting the references above the larger coefficient knee in a sorted plot of the set of all the crosscorrelations. 

1) The outliers may be found by comparing the crosscorrelation of each device with the pair of devices with the highest correlation coefficients. 

2) Compute the Allan deviation of the mean of all P subsets of N, 1< P <= N,  the drift is a minimum for the set of size P whose mean has the most constant Allan deviation 

3) Compute the Allan deviation of the difference of the individual references and the mean of the most stable group

QED
Reg

Actually easier than I thought.  But rather intimidating on first reading.  A *very* good question.

Edit: NB: The preceding analysis has nothing to do with sparse L1 pursuits.  It is a classical analysis based on the work of Norbert Wiener.  The best reference on the topic is "Random Data" by Bendat and Piersol.   I can think of no monograph on applied mathematics which is more useful for practical work.  I have a substantial fraction of the classic works on time series analysis.  Bendat and Piersol are superior by an order of magnitude.

Edit 2:  I should point out that the above assumed that the series were all of the same length.  If they are not of the same length, the statements still apply, but correctly computing the crosscorrelation coefficients becomes more nuanced.  This is particularly the case for long series for which using the FFT is desirable.  One of these nuances handed my head to me on a platter 20 years ago.  It was a very anxious week or so before I figured out what was going on.

[Svgeesus]

But an L1 solution, if it exists, has been proven to be identical if the columns of A are uncorrelated in all combinations and the x vector in x of Ax-y are sparse, that is, most of the coefficients of x are zero.

To put this in concrete terms, suppose we know from testing that the value of a resistor is f(a,b,c)  where:
a=initial value
b= change due to ambient temperature
c=change due to current flowing through the resistor

The constraint is that the columns are uncorrelated, but your example has two heating terms. Doesn't the heating due to current affect the local, ambient temperature?

[rhb]

But an L1 solution, if it exists, has been proven to be identical if the columns of A are uncorrelated in all combinations and the x vector in x of Ax-y are sparse, that is, most of the coefficients of x are zero.

To put this in concrete terms, suppose we know from testing that the value of a resistor is f(a,b,c)  where:
a=initial value
b= change due to ambient temperature
c=change due to current flowing through the resistor

The constraint is that the columns are uncorrelated, but your example has two heating terms. Doesn't the heating due to current affect the local, ambient temperature?

A very good question.  Solutions of the heat equation, a*F(kt) are only very slightly correlated for different values of k.  The ambient temperature term is not correlated with the self heating term. However, measurement during warmup is essential to solving for c.  Once the self heating approaches steady state they cannot be separated.

 I spent a good bit of time investigating the correlation of solutions of the heat equation  long before I learned about sparse L1 pursuits. I studied sparse L1 pursuits because I had been taught you could not solve the problems I had been solving.

[zhtoor]

Professor zhtoor :-)

If we measure the output of N references over a sufficiently long period of time and crosscorrelate the N references with the N-1 other references, the random noise will have zero correlation.  Therefore the references with the largest crosscorrelation coefficient will have the most similar drift.  This result is independent of the thermal noise of the switches and resistors.  So it applies even if those are non-zero.

The selection of the best set of references may be done by sorting the references by crosscorrelation coefficient and selecting the references above the larger coefficient knee in a sorted plot of the set of all the crosscorrelations. 

1) The outliers may be found by comparing the crosscorrelation of each device with the pair of devices with the highest correlation coefficients. 

2) Compute the Allan deviation of the mean of all P subsets of N, 1< P <= N,  the drift is a minimum for the set of size P whose mean has the most constant Allan deviation 

3) Compute the Allan deviation of the difference of the individual references and the mean of the most stable group

QED
Reg

Actually easier than I thought.  But rather intimidating on first reading.  A *very* good question.

Edit: NB: The preceding analysis has nothing to do with sparse L1 pursuits.  It is a classical analysis based on the work of Norbert Wiener.  The best reference on the topic is "Random Data" by Bendat and Piersol.   I can think of no monograph on applied mathematics which is more useful for practical work.  I have a substantial fraction of the classic works on time series analysis.  Bendat and Piersol are superior by an order of magnitude.

Edit 2:  I should point out that the above assumed that the series were all of the same length.  If they are not of the same length, the statements still apply, but correctly computing the crosscorrelation coefficients becomes more nuanced.  This is particularly the case for long series for which using the FFT is desirable.  One of these nuances handed my head to me on a platter 20 years ago.  It was a very anxious week or so before I figured out what was going on.

thanks. i am no professor, actually far from being one.

what would be the an efficient (optimal?) way of achieving the above?

actually, this is an attempt to generalize the 12-ball weighing problem for voltage reference characterization.
the real problem is the "weighing procedure", ie; how long to measure, where are the thresholds?

best regards.

-zia