doing math, scaled up integer or float? and pre-processor stuff

[Simon]

I am working with the ADC on an AVR. I am trying to get a result in a number that makes sense so that defines can be written in say mV. So the easy sounding solution is to work in float arithmetic but my guess is that this will mean adding a library I may not have room for and lots of CPU cycles.

Alternatively I can work in scaled up 32 bit integers. To get the actual number in mV (or mA) bearing in mind any scaling due to say input resistor dividers I tried to do:

"ADC result" x "ADC reference voltage in mV" x "input stepdown ratio" / "ADC maximum count"

But this turns out inaccurate for low values due to the division. With 13 bits, a reference of 1177 and no hardware scaling anything less than a result of 7 will produce 0 and I will of course not be retaining my resolution as every 7 counts will produce just 1 mV/mA. When I use a voltage reduction of 40 and ADC result of 1 count will produce a result of 5.75 with 0.75 dropped so any final value counts between multiples of 5 or 6 will be useless as well.

I am forced to use 32 bit variable math anyway so wy not multiply everything by 1024 and then divide it by 1024 at the end.

("ADC result" x "ADC reference voltage in mV" x "input stepdown ratio" x 1024 / "ADC maximum count") / 1024

So for 1 count:

(1 x 1177mV x 1 x 1024 / 8191) / 1024 = 0.143 so back to square one.

So this means that I need to work in µV / µA resolution of calculations while inputting mV /mA and not re-divide by 1024. I use 1024 for obvious reasons: 2^10, as a power of 2 it will be faster to calculate or does this not matter with the hardware multiplier as I am now not dividing by this number.

The problem now is that I have exceeded my 32 bit integer limit. Even if I take out the possible multiplication by the stepdown ratio so that I get ADC value µV rather than input to the resistor divider µV I am at 9x10^9.

This now gives a single count resolution of 147 µV which is not far off but I have tipped into 64 bit integer math so may as well work at nV resolution 😂.

If I look at this in the wider context my ADC reference will be at most 5'000 mV. That means that given 13 bits of resolution to stay within a 32 bit integer I cannot multiply by more than 100 (104), either as input resistor ratio divider or as a simple multiplier to get rid of rounding errors. Of course I could just multiply by 100 anyway and then multiply the final result by the input divider ratio as I have achieved all the resolution I ever will with 32 bit math - 10µV math with a voltage reference of 5V and 13 counts.

Effectively to get the most accurate result I need to have some random multiplier that is 2^32 / ( ADC counts X ADC Vref). This will get the finest result possible and will cater to input ratios up to that value. I then need to multiply any thresholds (in mV) by this value or divide them by this value for outputting visually so that they are in the same scale and all this can be worked out in the pre-processor - ah but that does not do math.....

So this is where I also fell down and was running around putting UL on the end of numbers that were something like "50" in order to try and cast the math operations.

So how do I tell the pre-processor or the C compiler via the pre-processor defines what bit size to do the calculations in? Other that UL I don't know anything else.

I also started to resolve this by casting at least one variable in 32 bit or just the entire operation:

value = (uint32_t) (the whole math sequence as above) ;

Should I cast each variable?, or is that unnecessary and only going to slow the thing down?

Or to avoid all confusion do I just create a variable for any definition, the defines simply keep things neat and tidy in the header file.

Have I just recreated floating point math? or will this be more efficient as I am solving one scenario not the whole world?

[RandallMcRee]

Simon,
You are reinventing so-called fixed point math.
You might want to google it.  There a number of libraries available but typically it’s overkill, unless you are like simulating physics in a game controller or some such. The concepts are what you need.

So I did  exactly what you are mentioning for a 24 bit adc.  As a check I wrote all the arithmetic stuff twice once in fixed point and once in 64 bit double and made sure that all fixed routines agreed for 2^^24 inputs. Of course the 64 bit routines are not included in the microcontroller.

I can show some examples if you think it will help.

Randall

[nctnico]

If speed and code size aren't an issue just use float.

Other than that using fixed point math is mainly about retaining resolution. So multiply first and then divide. But you also need to make sure not to cause an overflow. Circling back to your problem: since the ADC range, reference voltage and input divider ratio are likely fixed, you can create a single number which represents the maximum value (in millivolts) for your input. Say you have a 10V input that translates to 10000mV and an ADC range of 8191 (13 bits). You can reduce the calculation to:

mVolt = (adc reading * 10000) / 8191. Worst case 10000 * 8191 = 82e6 which easely fits into a 32 bit integer. For as long as the number of milli-Volts (or whatever unit you are using) is great than the ADC range, this won't lose resolution. In this case 10000 is greater than 8191 so that requirement is met.

For better rounding you can change it to:
mVolt = (adc reading * 10000 + 4095) / 8191

[David Hess]

Fixed point math would be my first choice, but you need to keep track of the radix yourself.  (1) Shift the converter result up as far as possible consistent with overflow requirements.

If the result is suppose to be human readable, then there is a case for using binary coded decimal (BCD).

(1) Some C compilers have support for fixed point math and track the radix automatically, but they are nonstandard and only for DSP processors that I know of.

[Simon]

Simon,
You are reinventing so-called fixed point math.

I know, but I am hoping to be optimised for resolution and speed.

If speed and code size aren't an issue just use float.

Other than that using fixed point math is mainly about retaining resolution. So multiply first and then divide. But you also need to make sure not to cause an overflow. Circling back to your problem: since the ADC range, reference voltage and input divider ratio are likely fixed, you can create a single number which represents the maximum value (in millivolts) for your input. Say you have a 10V input that translates to 10000mV and an ADC range of 8191 (13 bits). You can reduce the calculation to:

mVolt = (adc reading * 10000) / 8191. Worst case 10000 * 8191 = 82e6 which easely fits into a 32 bit integer. For as long as the number of milli-Volts (or whatever unit you are using) is great than the ADC range, this won't lose resolution. In this case 10000 is greater than 8191 so that requirement is met.

For better rounding you can change it to:
mVolt = (adc reading * 10000 + 4095) / 8191

Yes exactly. All you have done is taken the reference voltage out which looses track of what the number means. What I am looking to do is take into account the current reference voltage and then see how much more I can multiply the value by to maximise resolution. This way I actually know the difference between the scaling factor and the true value in mV. All these numbers are #define'd so they can be multiplied together by the compiler in optimizations assuming the XC8 compiler will do this much optimization without microchip wanting their pound of flesh. One day I will get the GCC compiler working in MPLABX.

Oh yes adding half the dividing number to the value before dividing is a neat idea, if I get a remainder of "0.5" it will tip it to "1", nice one.

Currently

Code: [Select]

"ADC value" = (uint_32t)("ADC result" x "ADC reference voltage in mV" x "input stepdown ratio" / "ADC maximum count")

takes about 720 clock cycles on the ATmega 0-series or 45µs which is time that I have although on all six channels I am currently using that is just over 0.25ms ADC value is 16 bit.

[Psi]

Simon,
You are reinventing so-called fixed point math.

I know, but I am hoping to be optimised for resolution and speed.

Most of the time when people say that they don't actually need it.

Use a float until you're at the point where your code is too slow or big. Then consider moving away from float.
9 times out of 10 you will never get to that point.

If you find you're always getting to that point on every project then you're probably trying too hard to spec the cheapest MCU possible.
Unless you're making 10,000 of something don't bother about trying to save 10 cents by getting a smaller MCU.

[Simon]

Yes that is true as well. I run my main program loop every 100ms with the ADC sampling in the background so the current 0.25ms is no sweat. It's been an interesting exercise and something that I want to be ready to implement as soon as I need it so understanding my options is still a very good idea.

I will probably take the result to value conversion out of my ADC interrupt routine. I can put it in the idling loop to calculate one value per idle loop what the last ADC result's value is.

[iMo]

"ADC result" x "ADC reference voltage in mV" x "input stepdown ratio" / "ADC maximum count"

Precalc that to

ADC_result * K

where K = 0.143694299 = 49 / 341

Do ((uint64_t)ADC_result * 1000000 * 49) / 341

or

( ( (uint32_t)ADC_result * 1000000 ) / 341) * 49   (EDIT:  or   * 100000 with the 13bit ADC)

[nctnico]

Simon,
You are reinventing so-called fixed point math.

I know, but I am hoping to be optimised for resolution and speed.

If speed and code size aren't an issue just use float.

Other than that using fixed point math is mainly about retaining resolution. So multiply first and then divide. But you also need to make sure not to cause an overflow. Circling back to your problem: since the ADC range, reference voltage and input divider ratio are likely fixed, you can create a single number which represents the maximum value (in millivolts) for your input. Say you have a 10V input that translates to 10000mV and an ADC range of 8191 (13 bits). You can reduce the calculation to:

mVolt = (adc reading * 10000) / 8191. Worst case 10000 * 8191 = 82e6 which easely fits into a 32 bit integer. For as long as the number of milli-Volts (or whatever unit you are using) is great than the ADC range, this won't lose resolution. In this case 10000 is greater than 8191 so that requirement is met.

For better rounding you can change it to:
mVolt = (adc reading * 10000 + 4095) / 8191

Yes exactly. All you have done is taken the reference voltage out which looses track of what the number means. What I am looking to do is take into account the current reference voltage and then see how much more I can multiply the value by to maximise resolution. This way I actually know the difference between the scaling factor and the true value in mV. All these numbers are #define'd so they can be multiplied together by the compiler in optimizations assuming the XC8 compiler will do this much optimization without microchip wanting their pound of flesh.

Then put the values in a comment so you (and anyone else after you) can see where the number comes from.

But basically your problem is using a compiler which is crippled on purpose. How much is your time worth? And how much time do you want to spend on things which are non-issues when using a decent compiler? It might be a good time to start thinking about using a different microcontroller for which there is good support from (for example) GCC.

[voltsandjolts]

BTW, XC8 now includes avrgcc for the avr mcus, so this is 'gcc' we are talking about.

Edit:
Microchip use gcc for XC32, XC16 and XC8-AVR
Basic optimisations are free but they charge for higher level optimisation.
All source code is available so they comply with gpl license but the build environment is obviscated to hinder re-compilation.
For XC16 XC32 there is this medicine: https://github.com/cv007/XC3216
Not sure about XC8-AVR.

XC8-PIC uses LLVM I believe, so they can lock that down.

[Simon]

So how does it work? I had to download GCC specifically but then it would never work. GCC is open source so they cant charge for it.

[Simon]

Simon,
You are reinventing so-called fixed point math.

I know, but I am hoping to be optimised for resolution and speed.

If speed and code size aren't an issue just use float.

Other than that using fixed point math is mainly about retaining resolution. So multiply first and then divide. But you also need to make sure not to cause an overflow. Circling back to your problem: since the ADC range, reference voltage and input divider ratio are likely fixed, you can create a single number which represents the maximum value (in millivolts) for your input. Say you have a 10V input that translates to 10000mV and an ADC range of 8191 (13 bits). You can reduce the calculation to:

mVolt = (adc reading * 10000) / 8191. Worst case 10000 * 8191 = 82e6 which easely fits into a 32 bit integer. For as long as the number of milli-Volts (or whatever unit you are using) is great than the ADC range, this won't lose resolution. In this case 10000 is greater than 8191 so that requirement is met.

For better rounding you can change it to:
mVolt = (adc reading * 10000 + 4095) / 8191

Yes exactly. All you have done is taken the reference voltage out which looses track of what the number means. What I am looking to do is take into account the current reference voltage and then see how much more I can multiply the value by to maximise resolution. This way I actually know the difference between the scaling factor and the true value in mV. All these numbers are #define'd so they can be multiplied together by the compiler in optimizations assuming the XC8 compiler will do this much optimization without microchip wanting their pound of flesh.

Then put the values in a comment so you (and anyone else after you) can see where the number comes from.

But basically your problem is using a compiler which is crippled on purpose. How much is your time worth? And how much time do you want to spend on things which are non-issues when using a decent compiler? It might be a good time to start thinking about using a different microcontroller for which there is good support from (for example) GCC.

We are both suggesting the same thing. But I am separating the multiplier out. If the Vref is defined as is the total counts then this multiplier is just a define too. I am just keeping track of it. So when I put into my program a define for the threshold in my so any comparison will just multiply by the worked out scaler. If the compiler optimizes these calculations they wont be in the program at all....

[voltsandjolts]

If I understand correctly, you are sampling faster than you are using the data, i.e. a lot of samples are discarded.
The one good thing about integer math is that it's fast, so you can implement a simple filter in an interrupt routine without much overhead, and make some use of all those samples.
To avoid the overhead of division you can do something like this:

Code: [Select]

/* simple first order iir filter */
/* adc sample rate dt=15ms, use time constant tc=200ms */
/* sample rate must be suitable for the bandwidth = 1/(2*pi*tc) */
/* use 65536 as the divisor so division is simply discarding the low word */
#define MULTIPLIER 4915 /* dt / tc = 15ms / 200ms = 0.075 = 4915/65536 */
#define HIWORD(w) (*(((unsigned int*)&(w))+1))

void adc_interrupt(void) {
    static int32_t acc=0;
    int32_t j;

    j  = adc;           /* x(n) */
    j -= HIWORD(acc);   /* x(n) - y(n) */
    j *= MULTIPLIER;    /* k( x(n) - y(n) ) */
    acc += j;
    adc_filtered = HIWORD(acc); /* y(n+1) = y(n) + k( x(n) - y(n) ) */
}

Then in your background code use adc_filtered, and scale it as discussed above.

[Simon]

Oh yes there are plenty of samples to do further sampling on. Although I have already oversampled the 10 bit ADC to 13 bit accumulating 64 results and dropping the last 3 bits. But here I am trying to do math with the least effort.

I don't quite get the code you have written. I would have to look into the filtering technique but the 2 variables that are created in the interrupt routine, surely the static one needs to be a global volatile? or it will just always be "0" having just been created.

[madires]

http://geeklikemetoo.blogspot.com/2016/10/fixed-point-math-with-avr-gcc.html

[nctnico]

Oh yes there are plenty of samples to do further sampling on. Although I have already oversampled the 10 bit ADC to 13 bit accumulating 64 results and dropping the last 3 bits. But here I am trying to do math with the least effort.

That might not work the way you like it to work. I have run into a situation where oversampling didn't give me any extra resolution because there wasn't enough noise for the 10 bit ADC to work with. And then there are non-linearities and likely the ADC in the microcontroller you are using isn't great to begin with.

[Simon]

Well even if the oversampling does not work at 10 bit I have enough resolution. The 0 series mega's have hardware accumulation and the result ready interrupt fires after the 64 results are accumulated so it's painless to do, it sort of filters a bit and maybe gets me more resolution. I do not know if I have enough noise. I never bother to disable the digital buffers and given that the system powers a motor I suspect plenty of noise in the voltage an current readings.

[mariush]

Not sure why you'd use a voltage reference of 1117 mV when there's 2048mV (2¹¹) and 4096 mV (2¹²) references and would work well with 12 bit or higher ADCs.

Those would allow you to just use some shift operations instead of multiplications or divisions.

May also want to consider lookup tables, for example may do the math with shifts instead of floating points and  have 4 bits or 8 bits per value as correction to the result you calculate. Maybe you can even figure out a set of values that can be reused (for adc readings 128 to 256, reuse lookup table for 0..127 and optionally add a fixed value)

[Simon]

Not sure why you'd use a voltage reference of 1117 mV when there's 2048mV (2¹¹) and 4096 mV (2¹²) references and would work well with 12 bit or higher ADCs.

Those would allow you to just use some shift operations instead of multiplications or divisions.

May also want to consider lookup tables, for example may do the math with shifts instead of floating points and  have 4 bits or 8 bits per value as correction to the result you calculate. Maybe you can even figure out a set of values that can be reused (for adc readings 128 to 256, reuse lookup table for 0..127 and optionally add a fixed value)

well it was meant to be 1.2V but it is actually 1.177 by the multimeter. I'll look into that one later, the 5V the resistor divider is supplied from is 4.995V, I guess it is the impedance of the Vref input that is pulling it down a bit. It's not the end of the world as long as it is consistent.

For the division by 8191 I will make it a division by 8192, or a 13 bit shift to the right having added 8192 to the number I am "dividing"

is your suggestion of a lookup table for the fractional parts that would be lost?

[Simon]

And then there are non-linearities and likely the ADC in the microcontroller you are using isn't great to begin with.

https://ww1.microchip.com/downloads/en/appnotes/atmel-2559-characterization-and-calibration-of-the-adc-on-an-avr_applicationnote_avr120.pdf

By the sounds of it not worth worrying too much about non linearities. The offset and gain can be easily compensated for but after that not worth worrying. They seemed to avoid talking about temperature compensation.

It sounds like the opamps in the AVR used for differential measurements are shit which hardly surprises me.

[T3sl4co1l]

Why do the units matter at all?  The ADC just measures a ratio to VREF.  All the math inside can be dimensionless and you don't have to worry about scale until the very end when printing to the user.

Starting with a 10 or 12 bit ADC, you may have enough fractional capacity in a 16 bit variable to do useful work.  I did this on my reverb effect box (XMEGA based).  16 bits was a bit tight, making some overflow tricky to handle (implementing saturating arithmetic rather than wrapping overflow; I was also using signed arithmetic to simplify summing/filtering operations).  Some operations were done with a 24 or 32 bit accumulator; you won't gain anything in C using 24 bits (there is a (u)int24_t primitive available in avr-gcc, but it's basically cast to 32-bit for all arithmetic operations) so just stick to 32 when you need more.

And yes indeed, you can do fixed point by implicitly multiplying everything by some [bit] offset; be very careful to remove the excess bits when multiplying, and make sure arithmetic is done with enough bits to fit the result before shifting: to C, it is absolutely natural to multiply two 16-bit variables into a 16-bit result, as absurd as that is mathematically.  In particular, byte shifts are basically free (GCC is awful at optimizing them, but again, unless you're doing ASM, it's just what you have to live with).

So, for example the product of two 8.8 fixed point variables, as 8.8 result, must be written:

Code: [Select]

int16_t fracmul(int16_t a, int16_t b) {
return ((int32_t)a * b) >> 8;
}
but you should also check for overflow to implement saturating arithmetic.

Like uh, let me see,

Code: [Select]

int16_t fracmul(int16_t a, int16_t b) {
int32_t p = (int32_t)a * b;
if (p > 0x00ffffff) return 0x7fff;
if (p < -0x00ffffff) return 0x8000;
return p >> 8;
}
There are better ways to do it (the compiler may realize the low bytes don't need to be checked, but low optimization will do the full four-byte comparison verbatim) but this should at least be correct.  (I'm quite proud of the few-CPU-cycle solution I found, in my assembler code, but it's special-casey of course and not something you can do in GCC alone.)

Tim

[PlainName]

surely the static one needs to be a global volatile? or it will just always be "0" having just been created.

A static variable in a function means it's on the heap rather than stack (so a 'global' that can only be seen by that function). The '0' is simply the initial value and is only set at program boot, not every time.

[nctnico]

Why do the units matter at all?  The ADC just measures a ratio to VREF.  All the math inside can be dimensionless and you don't have to worry about scale until the very end when printing to the user.

That is horrible for diagnostics and debugging. It is extremely handy & efficient to have the ADC value in mV or some other unit you are measuring because you can relate the value (you get from debug output and/or by watching a variable through a debugger) 1 on 1 with what is applied to the ADC externally. Using meaningful units takes away a source of error and confusion in a situation where a circuit and/or piece of software isn't behaving as it should OR during verification.

[David Hess]

Why do the units matter at all?  The ADC just measures a ratio to VREF.  All the math inside can be dimensionless and you don't have to worry about scale until the very end when printing to the user.

That is horrible for diagnostics and debugging. It is extremely handy & efficient to have the ADC value in mV or some other unit you are measuring because you can relate the value (you get from debug output and/or by watching a variable through a debugger) 1 on 1 with what is applied to the ADC externally. Using meaningful units takes away a source of error and confusion in a situation where a circuit and/or piece of software isn't behaving as it should OR during verification.

I agree with T3sl4co1l.  The meaningful units are "counts".

[JPortici]

BTW, XC8 now includes avrgcc for the avr mcus, so this is 'gcc' we are talking about.

Edit:
Microchip use gcc for XC32, XC16 and XC8-AVR
Basic optimisations are free but they charge for higher level optimisation.
All source code is available so they comply with gpl license but the build environment is obviscated to hinder re-compilation.
For XC16 XC32 there is this medicine: https://github.com/cv007/XC3216
Not sure about XC8-AVR.

XC8-PIC uses LLVM I believe, so they can lock that down.

-O2 is not enough? Only -O3 -Os and their own things (functional safety certified compiler/code-stack analysis/profiling) are paid options
soft floating point should be already "optimized" anyway, if not the compiler writers should be informed.