Electronics > Projects, Designs, and Technical Stuff

4th order polynomial coefficients for pressure at temperature readings, Help!!!

<< < (10/12) > >>

itsbiodiversity:
you are a MAD PERSON!  Wow.  Thanks so much for sticking with me on this.

IanB:

--- Quote from: itsbiodiversity on August 21, 2020, 11:27:48 am ---Those are the coefficients NOMINAL ANIMAL. Exactly. Now I need to figure out how to calculate those myself based on any dataset using excel.  I tried to download Gnuplot but had some issues installing.

--- End quote ---

As I mentioned in an earlier post, this is easily done using the Excel solver add-in (it is included with every version of Excel, but it does not show up on ribbon by default--you have to add it).

With the solver add-in you can say "vary these cells to minimize the value of that cell". It becomes quite easy to do all sorts of regressions that way.

Nominal Animal:
If you want to explore the gradient descent min-max optimizer, you can contact me via email, or continue discussion here.
(I'm only interested in this as an optimization problem, not looking to get anything.)
I noticed that I can get well over 50% acceleration using explicit AVX2 intrinsics; also a thread pool approach should work really well here.
My limitation is that I don't do Windows-only code.

In particular, if you have a calibration setup where you have more than one known good setup measuring the same thing as the device being calibrated, you could probably make the entire calibration process continuous.

b_force:
Noble amount of labor to figure all these things out  :-+

But I don't want to be much of a party pooper, but there are quite some programs available that can do very complicated curve fitting.

Nominal Animal:
Note that the gradient descent optimization is not curve fitting, it is actually an optimizer that works on the full parameter space, finding the local optimum (minimizing maximum error to calibration samples) from a given starting point.

Mathematically, we have a function \$f(a_0, a_1, a_2, a_3, a_4)\$ that provides the maximum error among the calibration points when the compensation function for measured pressure \$p\$ and temperature count \$c\$ is \$P(p, c) = a_0 + a_1 c + a_2 p + a_3 p c + a_4 p c^2\$.  The program assumes that only one calibration point causes the maximum error, and uses the gradient of that point – i.e., \$\left(\frac{\partial f}{\partial a_0}, \frac{\partial f}{\partial a_1}, \frac{\partial f}{\partial a_2}, \frac{\partial f}{\partial a_3}, \frac{\partial f}{\partial a_4} \right)\$ – to determine the direction in which to move in the parameter space.  The length of the step is first maximized (to the maximum length that provides a smaller maximum error than the starting point), then a binary search along this hyperline segment is used to find the \$(a_0, a_1, a_2, a_3, a_4)\$ that yields the smallest maximum error along this hyperline segment.

The program does a very, very large number of these descents.  It uses each unique quintet among the calibration samples (each unique 5-tuple of calibration points) to use the \$\left(\begin{matrix}N\\5\end{matrix}\right) = N(N-1)(N-2)(N-3)(N-4)/(1\cdot 2 \cdot 3 \cdot 4 \cdot 5)\$ starting points for the descents.  (Because of numerical instability of the naive implementation of the 5-point solver, it actually uses each of the 120 variants.)

In essence, the curve \$P(p, c) = a_0 + a_1 c + a_2 p + a_3 p c + a_4 p c^2\$ is only used with sets of 5 calibration samples, and then only to get useful starting points for the gradient descent.  There is no fitting at all, because the set of 5 samples define the five unknowns exactly.  (The code implements this algebraic solver itself.)

To get similar results from curve fitting, you use the maximum error in magnitude, \$\max\left(\left\lvert P(p_i,c_i) - A_i\right\rvert\right)\$ across the \$i\$ calibration samples (\$A_i\$ being the actual pressure being measured, \$p_i\$ the corresponding uncompensated measured pressure, and \$c_i\$ the temperature count).  But, since \$\max\left(\left\lvert P(p_i,c_i) - A_i\right\rvert\right)\$ is nonlinear (piecewise quadratic with a cross term), with a non-contiguous second derivative (because the \$i\$ that causes the maximum error varies), with LOTS of local minima (thousands if not more), normal methods (including anything based on linear algebra) will not produce as good results as this gradient descent does.

No need to believe me, though.  Just use the curve fitting package you think does better or even just as well, and see if it discovers the minimum I reported a couple of messages back, that minimizes the maximum error among the calibration samples to 0.002421 pressure units.  In comparison, least squares fitting only reaches a minimum error a magnitude larger (around 0.02, depending on details).

(Actually, if you can, please do; I would appreciate the results.  The ones I have access to – gnuplot, maxima, sagemath, maple, and mathematica – all use least squares fitting, which produces the same minimum error, about 0.02; or an order of magnitude worse maximum error than the gradient descent does.)

Navigation

[0] Message Index

[#] Next page

[*] Previous page

There was an error while thanking
Thanking...
Go to full version
Powered by SMFPacks Advanced Attachments Uploader Mod