Calibration - How to Mathematically

[jboard146]

I'm trying to wrap my head around the math around calibration tables, but don't know where to start. Google and ChatGPT aren't really a help probably due to my search/prompt.

For example if characterized a temperature sensor (it could be any sort of measured values). So i have a list of input and output values, but they are not linear. My real world example is measuring RF power which is a whole other rabbit hole so lets stick to a temperature sensor.

I'm assuming that when you have a calibration table you are trying to fit them into a linear line and not some polynomial function. Then code this into some sort of micro controller etc

I may be totally off here, but i think that is what i want to do.

I'm looking for a good reference book, video, web etc and/or what is this called in mathematics. A generic code example would also be great as well.

[calzap]

Would be helpful if you post an example of a calibration table and gave more detail about what you want to calibrate.   The basic mathematical approach to finding a function to approximate tabular data is called curve fitting.   Choice of function to fit can be via doing a plot of the data and deciding what function to use based on the apparent trend.   Even better is if you know the underlying mechanism the data (i.e., table values) represent.  The mechanism may align with a particular function.  A straight line is often not satisfactory.

Mike

[TimFox]

One common mathematical function to use in curve fitting is a polynomial, such as
y(x) = a₀ + a₁x + a₂x² + a₃x³
where the as are the calibration constants.
This can work well to interpolate between the measured calibration values, but such a polynomial will always "turn around" outside the range used for the measured data, so it should not be used to extrapolate past that range.

[brucehoult]

I'm assuming that when you have a calibration table you are trying to fit them into a linear line and not some polynomial function. Then code this into some sort of micro controller etc

Linear would be nice, for sure. Over a sufficiently small range any smooth function can be treated as linear.

But if you have to deal with a wide range of conditions and the physics aren't linear -- then you either accept inaccuracy, or you use a more complex function.

If you're lucky then quadratic or cubic might be close enough.  Piecewise cubic lets you match the slope as well as the value at both ends of each segment. But piecwise linear might be good enough.

You might want to approximate the inverse function rather than trying to solve the forward function.

Even a 16 MHz 8 bit AVR can do a lot of maths for each data point if if it only needs to do it 10 or 100 times a second. But these days the term "microcontroller" can include a 300 MHz 32 bit chip with hardware floating point.

[Picuino]

You can learn about splines too.
https://en.m.wikipedia.org/wiki/Spline_(mathematics)

[Nominal Animal]

I use wxMaxima or Maxima and gnuplot for most of my curve fitting needs.  They are free and open source, and available for all operating systems.  wxMaxima is the graphical user interface for Maxima, a computer algebra system.  Gnuplot is a tool for generating various kinds of plots, but can also do least-squares one-dimensional curve fitting (using nonlinear least-squares Marquardt-Levenberg algorithm) quite well.

If you have N values –– that is, N pairs of x and f(x) ––, then an N-1'th degree polynomial gives an exact fit.
For example, let's say you have a table of four points:
    x   f(x)
    25   100
    50   120
    75   145
    100  185
This tells us a third degree polynomial can describe this exactly.  In Maxima, we can find it out using
    f(x) := C0 + C1*x + C2*x^2 + C3*x^3;
    solve([ f(25) = 100, f(50) = 120, f(75) = 145, f(100) = 185 ], [ C0, C1, C2, C3 ]);
Note that there are four coefficients C0 through C3.  This is plain linear algebra, and by hand the solution is obtained by solving the system of equations, often in matrix form.  However, Maxima will tell us the solution coefficients:
    [[ C0 = 75, C1 = 37/30, C2 = -3/250, C3 = 1/9375 ]]
If you want them in decimal form, use for example float(%); which repeats the results as
    [[ C0 = 75.0, C1 = 1.23333333333, C2 = -0.012, C3 = 1.066666667e-4 ]]
Thus, the function we are looking for is exactly
    f(x) := 75 + 37*x/30 - 3*x*x/250 + x*x*x/9375;
To test, if you try f(25); f(50); f(75); f(100);, you will get the expected results, 100, 120, 145, and 185, respectively.  f(20) will yield 2393/25, or (after float(%);) 95.72.

You can use standard functions in the function definition, in which case it becomes a system of N equations with N unknowns, which may or may not have a solution or more than one solution.  If Maxima responds with just [], it means it couldn't find an exact solution.  If may also return several solutions, with %1, %2 as free variables (i.e., any value for them yields a valid solution). For example, if you ask
    solve(x^2 - 4 = 0, x);
it will tell you the two solutions,
    [x = -2, x = 2]
If I had the dosh, I might buy a license for Maple, because its optimizer and solver are slightly better, and I often play with annoyingly complex functions where I need to do transforms myself to get Maxima to find a solution.  Back in my Uni days, I loved to use Maple for that stuff (molecular dynamics potential models, forcefields, et cetera).

When you have many more data points to fit to, an exact function is often not realistic.  Instead, you want an approximate function, that reproduces the data points as close as possible.  The trick is how to define that "close", when you have a set of distances, and different methods for measuring the distance.  A typical definition of the distance is residual, which is just the difference between the fitted function value and the data set value for each data set point x.  (That means, the distance is only measured along the f(x) axis, or vertically in typical plots.)  A typical definition for "as close as possible" is least squares, which finds the fit that yields the smallest sum of each residual squared, for each of the given data points.  This is what is called "least squares fitting".

Let's say we wanted to fit f(x) = C0 + C1*x + C2*x*x to the above dataset using least squares fitting.  It cannot exactly reproduce the data points, but it should get close.  With gnuplot, we first prepare the data set, by creating a plain text file with one data point per line.  (Lines that begin with a # are comment lines, and gnuplot will ignore those.)  For example, let's say we create data.txt with
    # x f(x)
    25  100
    50  120
    75  145
    100 185
The first line is just a comment line for us humans.  We then start gnuplot, and tell it:
    f(x) = C0 + C1*x + C2*x*x
    fit f(x) "data.txt" using 1:2 via C0, C1, C2
The 1:2 there means first column for values of x, and second column for values of f(x).  It iterates for a while, then describes the fit it generated.  The fit is not the last lines, it is actually a bit before under the "Final set of parameters" heading, but it also sets the fit constants.  So, just run
    print C0, C1, C2
and it will tell you
    92.4999999999093 0.120000000003356 0.00799999999997438
i.e. the fit function is f(x) = 92.5 + 0.12*x + 0.008*x*x.
With gnuplot, we can now compare our fit to the points by plotting it as a graph:
    set xrange [ 0 : 125 ]; plot f(x) notitle with lines lc "red", "data.txt" using 1:2 notitle with points lc "black"
Similarly, we can print the fit function values in the four interesting points via
    print f(25), f(50), f(75), f(100)
which tells us the corresponding values (100.5, 118.5, 146.5, 184.5, so not exactly right, but not that much off either).

We could also just try a linear fit, via
    f(x) = C0 + C1*x ; fit f(x) "data.txt" u 1:2 via C0, C1
which is not too bad either.

If you want more control over fitting, say adding importance/weighing to the data points you have (and thus exceed basic "least squares" fitting with your own "distance" and "close enough" definitions), you can use Octave/Matlab (Octave being pretty compatible with Matlab but free and open source), or if you are familiar with Python, use numpy[/tt]/[URL=https://scipy.org/]scipy for numerical fitting (and other stuff, including Fourier transforms and such).  Statistics people tend to use R programming language, which is also free and open source, where fitting curves slots under the heading regression analysis, often linear regression (typical example is seeing a point cloud, and the linear regression line through it, showing the dependence between the point x and y coordinates).

For microcontroller stuff, the above is typically the starting point and not the final word, because they (especially Maxima) assume infinite-precision variables.  So, I personally often end up just finding the approximate values for the coefficients C0, C1, and so on, and write a brute-force (possibly aided by gradient descent at highest supported precision and range, typically long double on my machine) C program to find the exact values in the actual format I will use –– be that fixed point or float/IEEE 754 binary32 or double/IEEE 754 binary64 ––, that optimizes the approximate coefficients for the used numeric format.  This is often seen as silly or going too far, but writing the program is quick, and I can usually leave the computer to chug through the solution space overnight, so I don't see the silliness.  You might wish to check out the 4th order polynomial coefficients for pressure at temperature thread from a few years back for what that kind of optimization can be useful for, compared to just curve fitting.

[Picuino]

With chebyshev coefficients you can fit a function optimally, for minimize absolute error.
https://en.m.wikipedia.org/wiki/Chebyshev_nodes

https://en.m.wikipedia.org/wiki/Chebyshev_polynomials

[jpanhalt]

Yes, polynomials are one approach.  Their utility may depend on the math functions your MCU has.

If Maxim is a good role model, it uses a look-up table (LUT) for its MAX31856 and similar chips for thermocouples.  AMS also uses a LUT for its lightning detector ("Franklin") chip. I'm just an aged hobbyist and am not embarrassed to use a LUT to convert BCD mm to binary inches.  It's hard to beat a LUT for speed, and in my experience, if not overdone, they often take up less code space in the end.

[jboard146]

Thank you!
I at least know what i need to start reading about, which was the intent of my post.

[Nominal Animal]

Linear look-up and piecewise-defined functions via look-up tables can be both extremely fast and produce really good results for y=f(x).

The approach I like most is one where you select x so that they are densest where f(x) changes behaviour, with each pair sorted by x.  Then, a binary search can find the piece containing the x you want to find f(x) for very, very fast, and you need fewer of them than with a regular x spacing.

Linear interpolation is the easiest.  If you have \$x_0 < x < x_1\$, and \$y_0 = f(x_0)\$, \$y_1 = f(x_1)\$, then
$$y = f(x) = \frac{(x_1 - x) y_0 + (x - x_0) y_1}{x_1 - x_0}$$
This is numerically stable, and even with limited-precision numbers, will approach \$y_0\$ when \$x\$ approachex \$x_0\$, and \$y_1\$ when \$x\$ approaches \$x_1\$.  When using integer \$x\$ and \$y\$, that is the formula you'll want to use, even though it does involve a division.

You do not want to use \$f(x) = y_0 + (x - x_0) C\$ or \$f(x) = A + x B\$, where precomputed \$C = (y_1 - y_0) / (x_1 - x_0)\$ or \$A = (x_1 y_1 - x_0 y_0)/(x_1 - x_0)\$ and \$B = (y_0 - y_1) / (x_1 - x_0)\$, because these are not numerically stable.  Finite precision will cause a discontinuous jump when \$x\$ reaches \$x_1\$.  It is much better to use
$$y = f(x) = (x_1 - x) a + (x - x_0) b, \quad a = \frac{y_0}{x_1 - x_0}, \quad b = \frac{y_1}{x_1 - x_0}$$
saving the value of \$a\$ and \$b\$ for each \$x\$.  Note that you don't then need to save the values of \$y\$ at all, so it is just two constants for each value of \$x\$.

For cubic numerically stable interpolation, use
$$y = f(x) = (x - x_0)^3 C_0 + (x - x_0)^2 (x_1 - x) C_1 + (x - x_0) (x_1 - x)^2 C_2 + (x_1 - x)^3 C_3$$
where
$$C_0 = \frac{y_1}{(x_1 - x_0)^3}, \quad C_3 = \frac{y_0}{(x_1 - x_0)^3}, \quad C_1 = 3 C_0 - \frac{Y_1}{(x_1 - x_0)^2}, \quad C_2 = 3 C_3 + \frac{Y_0}{(x_1 - x_0)^2}$$
and \$Y_0\$ is the slope of \$f(x)\$ at \$x = x_0\$, and \$Y_1\$ is the slope of \$f(x)\$ at \$x = x_1\$.  Here, for each \$x\$, you save \$C_0\$, \$C_1\$, \$C_2\$, and \$C_3\$.  That is, if you have \$f(x)\$, and \$df(x)/dx\$ for a set of \$x\$, you can use cubic interpolation this way in a numerically stable fashion, saving just four constants \$C_0\$, \$C_1\$, \$C_2\$, and \$C_3\$ for each interval in \$x\$.
In your code, you look up the four constants (via a binary search, finding the largest \$x_0\$ smaller than your \$x\$, the next one being \$x_1\$, then calculate \$a_1 = x - x_0\$, \$a_2 = a_1^2\$, \$b_1 = x_1 - x\$, \$b_2 = b_1^2\$, and finally \$y = f(x) = a_1 a_2 C_0 + a_1 b_1 (a_1 C_1 + b_1 C_2) + C_3 b_1 b_2\$, for a total of 10 multiplications and 5 additions or subtractions.

[thermistor-guy]

I'm trying to wrap my head around the math around calibration tables, but don't know where to start. Google and ChatGPT aren't really a help probably due to my search/prompt.

For example if characterized a temperature sensor (it could be any sort of measured values). So i have a list of input and output values, but they are not linear. My real world example is measuring RF power which is a whole other rabbit hole so lets stick to a temperature sensor.

I'm assuming that when you have a calibration table you are trying to fit them into a linear line and not some polynomial function. Then code this into some sort of micro controller etc

I may be totally off here, but i think that is what i want to do.

I'm looking for a good reference book, video, web etc and/or what is this called in mathematics. A generic code example would also be great as well.

It's called approximation by interpolation, or just interpolation.

You have a function which maps input to output (sensor temperature to sensor resistance, say). You know the function at certain values (measurement points).
You want to estimate the function at values in between the measurement points (you want to "interpolate" the points).

If you want to purse the mathematical side, this takes you into fields called Approximation Theory, Optimal Recovery, and Functional Analysis.
I suggest you don't start there. It gets deep quickly.

One way to get started: pick up an introductory book on Numerical Analysis, and look for a chapter on Interpolation. Or look on-line for similar
material e.g.   https://en.wikipedia.org/wiki/Interpolation

Another way to get started: look for discussions of sensor linearization using look-up tables e.g.
    https://electronics.stackexchange.com/questions/412702/linearizing-a-0-10v-non-linear-sensor

[MrAl]

I'm trying to wrap my head around the math around calibration tables, but don't know where to start. Google and ChatGPT aren't really a help probably due to my search/prompt.

For example if characterized a temperature sensor (it could be any sort of measured values). So i have a list of input and output values, but they are not linear. My real world example is measuring RF power which is a whole other rabbit hole so lets stick to a temperature sensor.

I'm assuming that when you have a calibration table you are trying to fit them into a linear line and not some polynomial function. Then code this into some sort of micro controller etc

I may be totally off here, but i think that is what i want to do.

I'm looking for a good reference book, video, web etc and/or what is this called in mathematics. A generic code example would also be great as well.

Hi there,

As you know by now there are a ton of methods for doing this.  Some are more stable than others.  For example, a straight up polynomial curve fit does not always work out very good because there is no way to control the overshoots and undershoots and areas between data points.  To get this to work better you can introduce derivatives into that mix and get more control over how the fit works between data points.
The more general method is the Least Squares method which has been mentioned in this thread already.  A more advance version would be the Levenberg-Marquardt algorithm, but there are really a lot of choices, and there are some tricks you can use if you take a little more time to study the data.

Since this will be done with a microcontroller and you probably don't want it to take too long to compute each result, a common method is to just use linear interpretation between points in a table in which you store your measured and verified data.  I'll give a brief introduction.
Let's say you have inputs at x=1, 2, and 3, and corresponding outputs of 10, 20, and 30.  This would probably be a perfectly linear set but it doesn't have to be, so we will for now assume it is not really linear.  We will still use linear interpolation though as an example.
The thing with that data set is that we do not know what the output is for an input of say 1.2, 2.5, etc., we just know those three points.
To solve for the result with x=2.5 we know that x=3 minus x=2 is 1, and 1/2 of 1 is 0.5.  That's our scaling factor.  Since x=2 and x=3 corresponds to outputs of 20 and 30, we know the result lies between those two values.  Since the difference between those two is 10 and the scaling factor is 0.5, we can multiply 10 times 0.5 and we get 5.  Now we add that to 20 (the lower of 2 and 3) and we get 25.  That is the linear interpolation.
Now to get the solution for x=1.2 we do the same thing.  2 minus 1 is 1, and 1 times 0.2 is 0.2 which is the scaling factor now.  The output is between 10 and 20. and the difference is 10, and 10 times 0.2 is 2, and 10+2 equals 12, so the output is 12.  It's that simple and it's fast that is why it is used with the slower microcontrollers.
You just have to make sure you have enough data points so that you do not get large errors using this simple method.

There are some tricks you can use though.  With the example of the thermistor, if you take the natural log of each output first you can get a better fit.  That would mean storing values that are the natural log of the true values.  When you go to interpolate, you would take the constant 'e' up to the power of the table value to get the resistance.  If you use a fourth order polynomial combined with this method, you can get an accuracy of 0.1 percent over the entire temperature range of a 1 percent tolerance thermistor when compared to measured data.
You can also look up the formula for a thermistor and incorporate that.

The most common method though is to use a table and use linear interpolation.  If you do not have room to store a table though, you would have to resort to the Least Squares method, or the other one mentioned above.  The tradeoff is the time spent calculating the results.
There are also very intense algorithms out there for curve fitting that look at deviations and trends and other stuff in order to determine certain things about the fit and also about the formula being used for the fit, such as identifying outliers and redundant variables.  This gets very interesting.

[Muxr]

Wouldn't a simple sin(x)/x interpolation like the oscilloscopes do be simple and easy? If it's good enough for oscilloscopes it should be good enough for interpolating calibration points, no?

[Nominal Animal]

Wouldn't a simple sin(x)/x interpolation like the oscilloscopes do be simple and easy?

They exploit uniform sampling intervals; they don't actually calculate each point separately, but convert a sequence of regularly spaced samples \$x_i\$ to a much denser sequence of regularly spaced samples \$y_i\$.  It is pretty slow to calculate for a single arbitrary value, compared to the piecewise methods.

Cubic interpolation produces surprisinly good results, even for audio use.
Bézier curves, used in CAD and SVG and PostScript and PDF and basically everywhere, are cubic functions in each coordinate (with a curve parameter sweeping from 0 to 1).

[MarkT]

Bezier curves are parametric, which isn't the best fit for this purpose since we simply want to map x to y or y to x, rather than deal with x(t) and y(t).  Polynomial and rational approximations are commonly used.

[Nominal Animal]

Bezier curves are parametric, which isn't the best fit for this purpose since we simply want to map x to y or y to x, rather than deal with x(t) and y(t).

One-dimensional Bézier curve \$x(t) = (1-t)^3 X_0 + 3 (1-t)^2 t X_1 + 3 (1-t) t^2 X_2 + t^3 X_3\$ is strictly equivalent to a cubic polynomial \$y(t) = Y_0 + Y_1 t + Y_2 t^2 + Y_3 t^3\$ via \$Y_0 = X_0\$, \$Y_1 = 3 (X_1 - X_0)\$, \$Y_2 = 3 X_2 - 6 X_1 + 3 X_0\$, and \$Y_3 = X_3 - 3 X_2 + 3 X_1 - X_0\$, except that the Bézier form (with or without the \$3\$ constant factors for the middle terms) is numerically stable; compare to my post above, when you wish to interpolate any function for which you have some points \$x_k\$, the value at those points \$y_k = f(x_k)\$, and the slope (or derivative) at those points, \$Y_k = \lvert d f(x) / d x \rvert_{x = x_k}\$.

2D motion control via Bézier curves is very interesting, because the curves are split where the derivative changes sign (so that each leftover cubic curve consists only of segments \$t = 0 \dots 1\$ where the derivatives of each coordinate do not change sign), and perfect path is described by a sequence of bits where 0 indicates change in one axis, and 1 in the other.  It is then a matter of velocity control how fast this binary sequence is executed.  Funky, eh?  Only silly people do it by subdividing it into linear segments.

Point is, for univariate interpolation –– one dimension only, i.e. \$y = f(x)\$ –– you normally only use piecewise linear, piecewise cubic, or polynomials (or special dedicated functions) covering the entire range.

[thephil]

Many people already said helpful things about different approaches to fitting polynomials, piecewiese linear interpolation, splines etc. So all I would like to add is this: If you happen to know why and how the sensor response is non-linear, the best method is to fit the specific function that is rooted in the physical model instead of fitting a generic smoother. E.g. if you know the response is logarithmic, fit a log function, not a polynomial. But if you don't, go with splines/LOESS/polynomials/whatever.

[Picuino]

There is a problem with this approach. The curve may be approximately logarithmic, but not an exact logarithm. In that case (which is most of the time) it is again necessary to use a polynomial.

Note that logarithms are also calculated with polynomials, usually by doing the transformation:
t=(x-1)/(x+1)

And then applying a polynomial:
log_10(x) = a1·t + a3·t^3 + a5·t^5 + a7·t^7 + a9·t^9
a1=0.868591718
a3=0.289335524
a5=0.177522071
a7=0.094376476
a9=0.191337714
for x in the interval [1/sqrt(10), sqrt(10)]
(From Abramowitz and Stegun)

[Picuino]

Look at this code for logarithm, which uses a division of polynomials:

Code: [Select]

/*
   log returns the natural logarithm of its floating
   point argument.
   The coefficients are #2705 from Hart & Cheney. (19.38D)
   It calls frexp.
*/
#include <errno.h>
#include "cmath.h"

static const double p0 = -0.240139179559210510e2;
static const double p1 =  0.309572928215376501e2;
static const double p2 = -0.963769093368686593e1;
static const double p3 =  0.421087371217979714e0;
static const double q0 = -0.120069589779605255e2;
static const double q1 =  0.194809660700889731e2;
static const double q2 = -0.891110902798312337e1;

double log(double arg) {
   double x,z, zz, temp;
   int exp;

   if(arg <= 0.0) {
      errno = EDOM;
      return(-HUGE);
   }

   x = frexp(arg, &exp);
 
   if(x < INV_SQRT2) {
      x *= 2;
      exp--;
   }

   z = (x-1)/(x+1);
   zz = z*z;
   temp = ((p3*zz + p2)*zz + p1)*zz + p0;
   temp = temp/(((1.0*zz + q2)*zz + q1)*zz + q0);
   temp = temp*z + exp*LN2;
   return(temp);
}

double log10(double arg) {
   return(log(arg)*INV_LN10);
}

Only 7 coefficients and 18 decimal digits of precision!!!

[Picuino]

Rational polynomials are very accurate with few coefficients, but the coefficients are very difficult to compute.

Non-rational polynomials calculated with Chevyshev nodes are much simpler to calculate.
For the precision normally required in practical applications (3 to 5 decimal places) no more than 3 to 5 coefficients are usually necessary.



Example of function sen(x) approximated by polynomial with 4 digits of precision:

Code: [Select]

   xx = x * x;
   sin = 0.00761;
   sin = sin * xx - 0.16605;
   sin = sin * xx + 1;
   sin = sin * x;


Code: [Select]

x sin(x) p(x) relative error
0,00000 0,00000 0,00000 0,00000
0,06283 0,06279 0,06279 0,00000
0,12566 0,12533 0,12533 0,00001
0,18850 0,18738 0,18739 0,00002
0,25133 0,24869 0,24870 0,00004
0,31416 0,30902 0,30903 0,00005
0,37699 0,36812 0,36815 0,00008
0,43982 0,42578 0,42582 0,00010
0,50265 0,48175 0,48181 0,00012
0,56549 0,53583 0,53590 0,00014
0,62832 0,58779 0,58788 0,00015
0,69115 0,63742 0,63753 0,00016
0,75398 0,68455 0,68466 0,00017
0,81681 0,72897 0,72909 0,00017
0,87965 0,77051 0,77063 0,00015
0,94248 0,80902 0,80912 0,00013
1,00531 0,84433 0,84441 0,00010
1,06814 0,87631 0,87636 0,00006
1,13097 0,90483 0,90484 0,00002
1,19381 0,92978 0,92974 -0,00003
1,25664 0,95106 0,95097 -0,00009
1,31947 0,96858 0,96846 -0,00013
1,38230 0,98229 0,98213 -0,00016
1,44513 0,99211 0,99195 -0,00016
1,50796 0,99803 0,99791 -0,00012
1,57080 1,00000 1,00000 0,00000


[radiolistener]

Most easy and practical is to use quadratic interpolation with polynome. It requires 3 calibration points which give you 3 calibration coefficients. If you're needs to calibrate more complicated non linearity you can add more calibration points and use quadratic interpolation between each 3 points.

[Siwastaja]

Practical viewpoint: memory is nowadays cheap. Even the cheapest microcontrollers come with tens of kilobytes of flash memory which can be used to store calibration data.

So quite often a simple LUT for every possible value works out. For example, for 14-bit ADC input values producing 16-bit calibrated outputs, 2^14 * 2 = 32768-byte table is needed, which fits in modern day microcontrollers starting from the $1-class. As microcontrollers access the memory near to CPU processing speed anyway, random accessing memory is efficient. Advantages are simplicity of implementation, performance, and perfect behavior with weird nonlinearities which might not easily modeled with curve fitting.

If such large LUT nearly fits but not quite, then adding piecewise linear interpolation to reduce amount of data by maybe 2x or 4x is a good idea, but the further you decimate the worse it gets and then it's probably a better idea to use a polynomial fit, or for weird nonlinearities, a custom ad hoc function which models the system using a few parameters.

[DavidAlfa]

Try this, will make the polinomial based on input data.
You have most interpolation methods: Lagrange, Newton, Neville...

https://www.dcode.fr/function-equation-finder

[Picuino]

A practical example of the need for calibration is measure temperature with thermocouples.

From NIST:

Code: [Select]

ITS-90 Table for type T thermocouple                                           

 °C      0      1      2      3      4      5      6      7      8      9 
                               Thermoelectric Voltage in mV
 
   0  0.000  0.039  0.078  0.117  0.156  0.195  0.234  0.273  0.312  0.352
  10  0.391  0.431  0.470  0.510  0.549  0.589  0.629  0.669  0.709  0.749
  20  0.790  0.830  0.870  0.911  0.951  0.992  1.033  1.074  1.114  1.155
  30  1.196  1.238  1.279  1.320  1.362  1.403  1.445  1.486  1.528  1.570
  40  1.612  1.654  1.696  1.738  1.780  1.823  1.865  1.908  1.950  1.993
 
  50  2.036  2.079  2.122  2.165  2.208  2.251  2.294  2.338  2.381  2.425
  60  2.468  2.512  2.556  2.600  2.643  2.687  2.732  2.776  2.820  2.864
  70  2.909  2.953  2.998  3.043  3.087  3.132  3.177  3.222  3.267  3.312
  80  3.358  3.403  3.448  3.494  3.539  3.585  3.631  3.677  3.722  3.768
  90  3.814  3.860  3.907  3.953  3.999  4.046  4.092  4.138  4.185  4.232
 
 100  4.279  4.325  4.372  4.419  4.466  4.513  4.561  4.608  4.655  4.702
 110  4.750  4.798  4.845  4.893  4.941  4.988  5.036  5.084  5.132  5.180
 120  5.228  5.277  5.325  5.373  5.422  5.470  5.519  5.567  5.616  5.665
 130  5.714  5.763  5.812  5.861  5.910  5.959  6.008  6.057  6.107  6.156
 140  6.206  6.255  6.305  6.355  6.404  6.454  6.504  6.554  6.604  6.654
 
 150  6.704  6.754  6.805  6.855  6.905  6.956  7.006  7.057  7.107  7.158
 160  7.209  7.260  7.310  7.361  7.412  7.463  7.515  7.566  7.617  7.668
 170  7.720  7.771  7.823  7.874  7.926  7.977  8.029  8.081  8.133  8.185
 180  8.237  8.289  8.341  8.393  8.445  8.497  8.550  8.602  8.654  8.707
 190  8.759  8.812  8.865  8.917  8.970  9.023  9.076  9.129  9.182  9.235
 
 200  9.288  9.341  9.395  9.448  9.501  9.555  9.608  9.662  9.715  9.769
 210  9.822  9.876  9.930  9.984 10.038 10.092 10.146 10.200 10.254 10.308
 220 10.362 10.417 10.471 10.525 10.580 10.634 10.689 10.743 10.798 10.853
 230 10.907 10.962 11.017 11.072 11.127 11.182 11.237 11.292 11.347 11.403
 240 11.458 11.513 11.569 11.624 11.680 11.735 11.791 11.846 11.902 11.958
 
 250 12.013 12.069 12.125 12.181 12.237 12.293 12.349 12.405 12.461 12.518
 260 12.574 12.630 12.687 12.743 12.799 12.856 12.912 12.969 13.026 13.082
 270 13.139 13.196 13.253 13.310 13.366 13.423 13.480 13.537 13.595 13.652
 280 13.709 13.766 13.823 13.881 13.938 13.995 14.053 14.110 14.168 14.226
 290 14.283 14.341 14.399 14.456 14.514 14.572 14.630 14.688 14.746 14.804
 
 300 14.862 14.920 14.978 15.036 15.095 15.153 15.211 15.270 15.328 15.386
 310 15.445 15.503 15.562 15.621 15.679 15.738 15.797 15.856 15.914 15.973
 320 16.032 16.091 16.150 16.209 16.268 16.327 16.387 16.446 16.505 16.564
 330 16.624 16.683 16.742 16.802 16.861 16.921 16.980 17.040 17.100 17.159
 340 17.219 17.279 17.339 17.399 17.458 17.518 17.578 17.638 17.698 17.759
 
 350 17.819 17.879 17.939 17.999 18.060 18.120 18.180 18.241 18.301 18.362
 360 18.422 18.483 18.543 18.604 18.665 18.725 18.786 18.847 18.908 18.969
 370 19.030 19.091 19.152 19.213 19.274 19.335 19.396 19.457 19.518 19.579
 380 19.641 19.702 19.763 19.825 19.886 19.947 20.009 20.070 20.132 20.193
 390 20.255 20.317 20.378 20.440 20.502 20.563 20.625 20.687 20.748 20.810
A polinomial with this coefficients:

Code: [Select]

[-1.07895005e-14  2.95331121e-11 -4.34050298e-08  4.77945039e-05  3.83895381e-02  2.24050514e-03]
fits the table.


Done with this Python program:

Code: [Select]

import numpy as np

voltages = [
   0.000, 0.039, 0.078, 0.117, 0.156, 0.195, 0.234, 0.273, 0.312, 0.352,
   0.391, 0.431, 0.470, 0.510, 0.549, 0.589, 0.629, 0.669, 0.709, 0.749,
   0.790, 0.830, 0.870, 0.911, 0.951, 0.992, 1.033, 1.074, 1.114, 1.155,
   1.196, 1.238, 1.279, 1.320, 1.362, 1.403, 1.445, 1.486, 1.528, 1.570,
   1.612, 1.654, 1.696, 1.738, 1.780, 1.823, 1.865, 1.908, 1.950, 1.993,
   2.036, 2.079, 2.122, 2.165, 2.208, 2.251, 2.294, 2.338, 2.381, 2.425,
   2.468, 2.512, 2.556, 2.600, 2.643, 2.687, 2.732, 2.776, 2.820, 2.864,
   2.909, 2.953, 2.998, 3.043, 3.087, 3.132, 3.177, 3.222, 3.267, 3.312,
   3.358, 3.403, 3.448, 3.494, 3.539, 3.585, 3.631, 3.677, 3.722, 3.768,
   3.814, 3.860, 3.907, 3.953, 3.999, 4.046, 4.092, 4.138, 4.185, 4.232,
   4.279, 4.325, 4.372, 4.419, 4.466, 4.513, 4.561, 4.608, 4.655, 4.702,
   4.750, 4.798, 4.845, 4.893, 4.941, 4.988, 5.036, 5.084, 5.132, 5.180,
   5.228, 5.277, 5.325, 5.373, 5.422, 5.470, 5.519, 5.567, 5.616, 5.665,
   5.714, 5.763, 5.812, 5.861, 5.910, 5.959, 6.008, 6.057, 6.107, 6.156,
   6.206, 6.255, 6.305, 6.355, 6.404, 6.454, 6.504, 6.554, 6.604, 6.654,
   6.704, 6.754, 6.805, 6.855, 6.905, 6.956, 7.006, 7.057, 7.107, 7.158,
   7.209, 7.260, 7.310, 7.361, 7.412, 7.463, 7.515, 7.566, 7.617, 7.668,
   7.720, 7.771, 7.823, 7.874, 7.926, 7.977, 8.029, 8.081, 8.133, 8.185,
   8.237, 8.289, 8.341, 8.393, 8.445, 8.497, 8.550, 8.602, 8.654, 8.707,
   8.759, 8.812, 8.865, 8.917, 8.970, 9.023, 9.076, 9.129, 9.182, 9.235,
   9.288, 9.341, 9.395, 9.448, 9.501, 9.555, 9.608, 9.662, 9.715, 9.769,
   9.822, 9.876, 9.930, 9.984, 10.038, 10.092, 10.146, 10.200, 10.254, 10.308,
   10.362, 10.417, 10.471, 10.525, 10.580, 10.634, 10.689, 10.743, 10.798, 10.853,
   10.907, 10.962, 11.017, 11.072, 11.127, 11.182, 11.237, 11.292, 11.347, 11.403,
   11.458, 11.513, 11.569, 11.624, 11.680, 11.735, 11.791, 11.846, 11.902, 11.958,
   12.013, 12.069, 12.125, 12.181, 12.237, 12.293, 12.349, 12.405, 12.461, 12.518,
   12.574, 12.630, 12.687, 12.743, 12.799, 12.856, 12.912, 12.969, 13.026, 13.082,
   13.139, 13.196, 13.253, 13.310, 13.366, 13.423, 13.480, 13.537, 13.595, 13.652,
   13.709, 13.766, 13.823, 13.881, 13.938, 13.995, 14.053, 14.110, 14.168, 14.226,
   14.283, 14.341, 14.399, 14.456, 14.514, 14.572, 14.630, 14.688, 14.746, 14.804,
   14.862, 14.920, 14.978, 15.036, 15.095, 15.153, 15.211, 15.270, 15.328, 15.386,
   15.445, 15.503, 15.562, 15.621, 15.679, 15.738, 15.797, 15.856, 15.914, 15.973,
   16.032, 16.091, 16.150, 16.209, 16.268, 16.327, 16.387, 16.446, 16.505, 16.564,
   16.624, 16.683, 16.742, 16.802, 16.861, 16.921, 16.980, 17.040, 17.100, 17.159,
   17.219, 17.279, 17.339, 17.399, 17.458, 17.518, 17.578, 17.638, 17.698, 17.759,
   17.819, 17.879, 17.939, 17.999, 18.060, 18.120, 18.180, 18.241, 18.301, 18.362,
   18.422, 18.483, 18.543, 18.604, 18.665, 18.725, 18.786, 18.847, 18.908, 18.969,
   19.030, 19.091, 19.152, 19.213, 19.274, 19.335, 19.396, 19.457, 19.518, 19.579,
   19.641, 19.702, 19.763, 19.825, 19.886, 19.947, 20.009, 20.070, 20.132, 20.193,
   20.255, 20.317, 20.378, 20.440, 20.502, 20.563, 20.625, 20.687, 20.748, 20.810
]

x = np.array(list(range(0, 400)))
y = np.array(voltages[0:400])
print(x, y)
coef= np.polyfit(x, y, 5)
print(coef)

p = np.poly1d(coef)
for t in range(1, 400):
   print((p(t)-voltages[t])*1000)



References:
https://srdata.nist.gov/its90/main/
https://www.analog.com/en/resources/analog-dialogue/articles/measuring-temp-using-thermocouples.html

[Psi]

Here's a VERY basic linear lookup example, just in case that is useful to anyone overwhelmed by the complex formulas above.
 
Lookup table example (input, output)
1 = 10
20 = 500
30 = 1000

When your code has an input value and wants to calculate the output value lookup it loops through checking the lines until the next line's input value is larger or equal to the value its looking for. So, if the input value is 25, this check will become true when it checks line 2, because on the 3rd line we have 30 which is larger than 25.
So now we have located the two lookup table lines that are on each side of the value 25 that we are looking up.
eg
20 = 500
30 = 1000

Now we just need to do some linear interpolation to figure out what output value 25 will be based on the two values that we do have (20 and 30)
(Looking at the data we know this is going to be 750, because 25 is halfway between 20 and 30 and 750 is halfway between 500 and 1000.)
The math to calculate it is pretty simple by finding a percentage for the position on the input side and applying that to the output side.
30-20 = 10
25-20 = 5
5/10 = 0.5    (50%)
and on the output side
1000-500 = 500
0.5 * 500 = 250
250+500 = 750
Interpolation done.

Now this example is VERY basic and has some issues, but if you find the other examples in the thread too technical this one is easy to understand
and you can expand on it now that you can see the basics of how it works.