Author Topic: doing math, scaled up integer or float? and pre-processor stuff  (Read 6881 times)

0 Members and 1 Guest are viewing this topic.

Offline SimonTopic starter

  • Global Moderator
  • *****
  • Posts: 17816
  • Country: gb
  • Did that just blow up? No? might work after all !!
    • Simon's Electronics
doing math, scaled up integer or float? and pre-processor stuff
« on: September 04, 2021, 08:23:10 pm »
I am working with the ADC on an AVR. I am trying to get a result in a number that makes sense so that defines can be written in say mV. So the easy sounding solution is to work in float arithmetic but my guess is that this will mean adding a library I may not have room for and lots of CPU cycles.

Alternatively I can work in scaled up 32 bit integers. To get the actual number in mV (or mA) bearing in mind any scaling due to say input resistor dividers I tried to do:

"ADC result" x "ADC reference voltage in mV" x "input stepdown ratio" / "ADC maximum count"

But this turns out inaccurate for low values due to the division. With 13 bits, a reference of 1177 and no hardware scaling anything less than a result of 7 will produce 0 and I will of course not be retaining my resolution as every 7 counts will produce just 1 mV/mA. When I use a voltage reduction of 40 and ADC result of 1 count will produce a result of 5.75 with 0.75 dropped so any final value counts between multiples of 5 or 6 will be useless as well.

I am forced to use 32 bit variable math anyway so wy not multiply everything by 1024 and then divide it by 1024 at the end.

("ADC result" x "ADC reference voltage in mV" x "input stepdown ratio" x 1024 / "ADC maximum count") / 1024

So for 1 count:

(1 x 1177mV x 1 x 1024 / 8191) / 1024 = 0.143 so back to square one.

So this means that I need to work in µV / µA resolution of calculations while inputting mV /mA and not re-divide by 1024. I use 1024 for obvious reasons: 2^10, as a power of 2 it will be faster to calculate or does this not matter with the hardware multiplier as I am now not dividing by this number.

The problem now is that I have exceeded my 32 bit integer limit. Even if I take out the possible multiplication by the stepdown ratio so that I get ADC value µV rather than input to the resistor divider µV I am at 9x10^9.

This now gives a single count resolution of 147 µV which is not far off but I have tipped into 64 bit integer math so may as well work at nV resolution 😂.

If I look at this in the wider context my ADC reference will be at most 5'000 mV. That means that given 13 bits of resolution to stay within a 32 bit integer I cannot multiply by more than 100 (104), either as input resistor ratio divider or as a simple multiplier to get rid of rounding errors. Of course I could just multiply by 100 anyway and then multiply the final result by the input divider ratio as I have achieved all the resolution I ever will with 32 bit math - 10µV math with a voltage reference of 5V and 13 counts.

Effectively to get the most accurate result I need to have some random multiplier that is 2^32 / ( ADC counts X ADC Vref). This will get the finest result possible and will cater to input ratios up to that value. I then need to multiply any thresholds (in mV) by this value or divide them by this value for outputting visually so that they are in the same scale and all this can be worked out in the pre-processor - ah but that does not do math.....

So this is where I also fell down and was running around putting UL on the end of numbers that were something like "50" in order to try and cast the math operations.

So how do I tell the pre-processor or the C compiler via the pre-processor defines what bit size to do the calculations in? Other that UL I don't know anything else.

I also started to resolve this by casting at least one variable in 32 bit or just the entire operation:

value = (uint32_t) (the whole math sequence as above) ;

Should I cast each variable?, or is that unnecessary and only going to slow the thing down?

Or to avoid all confusion do I just create a variable for any definition, the defines simply keep things neat and tidy in the header file.

Have I just recreated floating point math? or will this be more efficient as I am solving one scenario not the whole world?

 

Online RandallMcRee

  • Frequent Contributor
  • **
  • Posts: 541
  • Country: us
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #1 on: September 04, 2021, 09:05:20 pm »
Simon,
You are reinventing so-called fixed point math.
You might want to google it.  There a number of libraries available but typically it’s overkill, unless you are like simulating physics in a game controller or some such. The concepts are what you need.

So I did  exactly what you are mentioning for a 24 bit adc.  As a check I wrote all the arithmetic stuff twice once in fixed point and once in 64 bit double and made sure that all fixed routines agreed for 2^^24 inputs. Of course the 64 bit routines are not included in the microcontroller.

I can show some examples if you think it will help.

Randall
 

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 26907
  • Country: nl
    • NCT Developments
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #2 on: September 04, 2021, 11:24:03 pm »
If speed and code size aren't an issue just use float.

Other than that using fixed point math is mainly about retaining resolution. So multiply first and then divide. But you also need to make sure not to cause an overflow. Circling back to your problem: since the ADC range, reference voltage and input divider ratio are likely fixed, you can create a single number which represents the maximum value (in millivolts) for your input. Say you have a 10V input that translates to 10000mV and an ADC range of 8191 (13 bits). You can reduce the calculation to:

mVolt = (adc reading * 10000) / 8191. Worst case 10000 * 8191 = 82e6 which easely fits into a 32 bit integer. For as long as the number of milli-Volts (or whatever unit you are using) is great than the ADC range, this won't lose resolution. In this case 10000 is greater than 8191 so that requirement is met.

For better rounding you can change it to:
mVolt = (adc reading * 10000 + 4095) / 8191
« Last Edit: September 04, 2021, 11:30:21 pm by nctnico »
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline David Hess

  • Super Contributor
  • ***
  • Posts: 16618
  • Country: us
  • DavidH
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #3 on: September 04, 2021, 11:53:59 pm »
Fixed point math would be my first choice, but you need to keep track of the radix yourself.  (1) Shift the converter result up as far as possible consistent with overflow requirements.

If the result is suppose to be human readable, then there is a case for using binary coded decimal (BCD).

(1) Some C compilers have support for fixed point math and track the radix automatically, but they are nonstandard and only for DSP processors that I know of.
« Last Edit: September 05, 2021, 08:32:58 am by David Hess »
 

Offline SimonTopic starter

  • Global Moderator
  • *****
  • Posts: 17816
  • Country: gb
  • Did that just blow up? No? might work after all !!
    • Simon's Electronics
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #4 on: September 05, 2021, 06:55:49 am »
Simon,
You are reinventing so-called fixed point math.


I know, but I am hoping to be optimised for resolution and speed.

If speed and code size aren't an issue just use float.

Other than that using fixed point math is mainly about retaining resolution. So multiply first and then divide. But you also need to make sure not to cause an overflow. Circling back to your problem: since the ADC range, reference voltage and input divider ratio are likely fixed, you can create a single number which represents the maximum value (in millivolts) for your input. Say you have a 10V input that translates to 10000mV and an ADC range of 8191 (13 bits). You can reduce the calculation to:

mVolt = (adc reading * 10000) / 8191. Worst case 10000 * 8191 = 82e6 which easely fits into a 32 bit integer. For as long as the number of milli-Volts (or whatever unit you are using) is great than the ADC range, this won't lose resolution. In this case 10000 is greater than 8191 so that requirement is met.

For better rounding you can change it to:
mVolt = (adc reading * 10000 + 4095) / 8191


Yes exactly. All you have done is taken the reference voltage out which looses track of what the number means. What I am looking to do is take into account the current reference voltage and then see how much more I can multiply the value by to maximise resolution. This way I actually know the difference between the scaling factor and the true value in mV. All these numbers are #define'd so they can be multiplied together by the compiler in optimizations assuming the XC8 compiler will do this much optimization without microchip wanting their pound of flesh. One day I will get the GCC compiler working in MPLABX.

Oh yes adding half the dividing number to the value before dividing is a neat idea, if I get a remainder of "0.5" it will tip it to "1", nice one.

Currently

Code: [Select]
"ADC value" = (uint_32t)("ADC result" x "ADC reference voltage in mV" x "input stepdown ratio" / "ADC maximum count")

takes about 720 clock cycles on the ATmega 0-series or 45µs which is time that I have although on all six channels I am currently using that is just over 0.25ms ADC value is 16 bit.
 

Offline Psi

  • Super Contributor
  • ***
  • Posts: 9951
  • Country: nz
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #5 on: September 05, 2021, 07:01:40 am »
Simon,
You are reinventing so-called fixed point math.
I know, but I am hoping to be optimised for resolution and speed.

Most of the time when people say that they don't actually need it.

Use a float until you're at the point where your code is too slow or big. Then consider moving away from float.
9 times out of 10 you will never get to that point.

If you find you're always getting to that point on every project then you're probably trying too hard to spec the cheapest MCU possible.
Unless you're making 10,000 of something don't bother about trying to save 10 cents by getting a smaller MCU.
« Last Edit: September 05, 2021, 07:07:52 am by Psi »
Greek letter 'Psi' (not Pounds per Square Inch)
 
The following users thanked this post: nctnico, Jacon

Offline SimonTopic starter

  • Global Moderator
  • *****
  • Posts: 17816
  • Country: gb
  • Did that just blow up? No? might work after all !!
    • Simon's Electronics
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #6 on: September 05, 2021, 07:09:04 am »
Yes that is true as well. I run my main program loop every 100ms with the ADC sampling in the background so the current 0.25ms is no sweat. It's been an interesting exercise and something that I want to be ready to implement as soon as I need it so understanding my options is still a very good idea.

I will probably take the result to value conversion out of my ADC interrupt routine. I can put it in the idling loop to calculate one value per idle loop what the last ADC result's value is.
 

Online iMo

  • Super Contributor
  • ***
  • Posts: 4789
  • Country: pm
  • It's important to try new things..
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #7 on: September 05, 2021, 07:46:32 am »
Quote
"ADC result" x "ADC reference voltage in mV" x "input stepdown ratio" / "ADC maximum count"

Precalc that to

ADC_result * K

where K = 0.143694299 = 49 / 341

Do ((uint64_t)ADC_result * 1000000 * 49) / 341

or

( ( (uint32_t)ADC_result * 1000000 ) / 341) * 49   (EDIT:  or   * 100000 with the 13bit ADC)

« Last Edit: September 05, 2021, 09:03:52 am by imo »
 

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 26907
  • Country: nl
    • NCT Developments
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #8 on: September 05, 2021, 10:03:43 am »
Simon,
You are reinventing so-called fixed point math.


I know, but I am hoping to be optimised for resolution and speed.

If speed and code size aren't an issue just use float.

Other than that using fixed point math is mainly about retaining resolution. So multiply first and then divide. But you also need to make sure not to cause an overflow. Circling back to your problem: since the ADC range, reference voltage and input divider ratio are likely fixed, you can create a single number which represents the maximum value (in millivolts) for your input. Say you have a 10V input that translates to 10000mV and an ADC range of 8191 (13 bits). You can reduce the calculation to:

mVolt = (adc reading * 10000) / 8191. Worst case 10000 * 8191 = 82e6 which easely fits into a 32 bit integer. For as long as the number of milli-Volts (or whatever unit you are using) is great than the ADC range, this won't lose resolution. In this case 10000 is greater than 8191 so that requirement is met.

For better rounding you can change it to:
mVolt = (adc reading * 10000 + 4095) / 8191

Yes exactly. All you have done is taken the reference voltage out which looses track of what the number means. What I am looking to do is take into account the current reference voltage and then see how much more I can multiply the value by to maximise resolution. This way I actually know the difference between the scaling factor and the true value in mV. All these numbers are #define'd so they can be multiplied together by the compiler in optimizations assuming the XC8 compiler will do this much optimization without microchip wanting their pound of flesh.
Then put the values in a comment so you (and anyone else after you) can see where the number comes from.

But basically your problem is using a compiler which is crippled on purpose. How much is your time worth? And how much time do you want to spend on things which are non-issues when using a decent compiler? It might be a good time to start thinking about using a different microcontroller for which there is good support from (for example) GCC.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline voltsandjolts

  • Supporter
  • ****
  • Posts: 2300
  • Country: gb
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #9 on: September 05, 2021, 10:19:01 am »
BTW, XC8 now includes avrgcc for the avr mcus, so this is 'gcc' we are talking about.

Edit:
Microchip use gcc for XC32, XC16 and XC8-AVR
Basic optimisations are free but they charge for higher level optimisation.
All source code is available so they comply with gpl license but the build environment is obviscated to hinder re-compilation.
For XC16 XC32 there is this medicine: https://github.com/cv007/XC3216
Not sure about XC8-AVR.

XC8-PIC uses LLVM I believe, so they can lock that down.
« Last Edit: September 05, 2021, 10:52:32 am by voltsandjolts »
 

Offline SimonTopic starter

  • Global Moderator
  • *****
  • Posts: 17816
  • Country: gb
  • Did that just blow up? No? might work after all !!
    • Simon's Electronics
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #10 on: September 05, 2021, 10:30:35 am »
So how does it work? I had to download GCC specifically but then it would never work. GCC is open source so they cant charge for it.
 

Offline SimonTopic starter

  • Global Moderator
  • *****
  • Posts: 17816
  • Country: gb
  • Did that just blow up? No? might work after all !!
    • Simon's Electronics
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #11 on: September 05, 2021, 10:35:40 am »
Simon,
You are reinventing so-called fixed point math.


I know, but I am hoping to be optimised for resolution and speed.

If speed and code size aren't an issue just use float.

Other than that using fixed point math is mainly about retaining resolution. So multiply first and then divide. But you also need to make sure not to cause an overflow. Circling back to your problem: since the ADC range, reference voltage and input divider ratio are likely fixed, you can create a single number which represents the maximum value (in millivolts) for your input. Say you have a 10V input that translates to 10000mV and an ADC range of 8191 (13 bits). You can reduce the calculation to:

mVolt = (adc reading * 10000) / 8191. Worst case 10000 * 8191 = 82e6 which easely fits into a 32 bit integer. For as long as the number of milli-Volts (or whatever unit you are using) is great than the ADC range, this won't lose resolution. In this case 10000 is greater than 8191 so that requirement is met.

For better rounding you can change it to:
mVolt = (adc reading * 10000 + 4095) / 8191

Yes exactly. All you have done is taken the reference voltage out which looses track of what the number means. What I am looking to do is take into account the current reference voltage and then see how much more I can multiply the value by to maximise resolution. This way I actually know the difference between the scaling factor and the true value in mV. All these numbers are #define'd so they can be multiplied together by the compiler in optimizations assuming the XC8 compiler will do this much optimization without microchip wanting their pound of flesh.
Then put the values in a comment so you (and anyone else after you) can see where the number comes from.

But basically your problem is using a compiler which is crippled on purpose. How much is your time worth? And how much time do you want to spend on things which are non-issues when using a decent compiler? It might be a good time to start thinking about using a different microcontroller for which there is good support from (for example) GCC.


We are both suggesting the same thing. But I am separating the multiplier out. If the Vref is defined as is the total counts then this multiplier is just a define too. I am just keeping track of it. So when I put into my program a define for the threshold in my so any comparison will just multiply by the worked out scaler. If the compiler optimizes these calculations they wont be in the program at all....
 

Offline voltsandjolts

  • Supporter
  • ****
  • Posts: 2300
  • Country: gb
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #12 on: September 05, 2021, 04:11:15 pm »
If I understand correctly, you are sampling faster than you are using the data, i.e. a lot of samples are discarded.
The one good thing about integer math is that it's fast, so you can implement a simple filter in an interrupt routine without much overhead, and make some use of all those samples.
To avoid the overhead of division you can do something like this:

Code: [Select]
/* simple first order iir filter */
/* adc sample rate dt=15ms, use time constant tc=200ms */
/* sample rate must be suitable for the bandwidth = 1/(2*pi*tc) */
/* use 65536 as the divisor so division is simply discarding the low word */
#define MULTIPLIER 4915 /* dt / tc = 15ms / 200ms = 0.075 = 4915/65536 */
#define HIWORD(w) (*(((unsigned int*)&(w))+1))

void adc_interrupt(void) {
    static int32_t acc=0;
    int32_t j;

    j  = adc;           /* x(n) */
    j -= HIWORD(acc);   /* x(n) - y(n) */
    j *= MULTIPLIER;    /* k( x(n) - y(n) ) */
    acc += j;
    adc_filtered = HIWORD(acc); /* y(n+1) = y(n) + k( x(n) - y(n) ) */
}

Then in your background code use adc_filtered, and scale it as discussed above.
« Last Edit: September 06, 2021, 11:08:33 am by voltsandjolts »
 
The following users thanked this post: Bassman59

Offline SimonTopic starter

  • Global Moderator
  • *****
  • Posts: 17816
  • Country: gb
  • Did that just blow up? No? might work after all !!
    • Simon's Electronics
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #13 on: September 05, 2021, 05:07:07 pm »
Oh yes there are plenty of samples to do further sampling on. Although I have already oversampled the 10 bit ADC to 13 bit accumulating 64 results and dropping the last 3 bits. But here I am trying to do math with the least effort.

I don't quite get the code you have written. I would have to look into the filtering technique but the 2 variables that are created in the interrupt routine, surely the static one needs to be a global volatile? or it will just always be "0" having just been created.
 

Offline madires

  • Super Contributor
  • ***
  • Posts: 7765
  • Country: de
  • A qualified hobbyist ;)
 

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 26907
  • Country: nl
    • NCT Developments
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #15 on: September 05, 2021, 06:15:19 pm »
Oh yes there are plenty of samples to do further sampling on. Although I have already oversampled the 10 bit ADC to 13 bit accumulating 64 results and dropping the last 3 bits. But here I am trying to do math with the least effort.
That might not work the way you like it to work. I have run into a situation where oversampling didn't give me any extra resolution because there wasn't enough noise for the 10 bit ADC to work with. And then there are non-linearities and likely the ADC in the microcontroller you are using isn't great to begin with.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline SimonTopic starter

  • Global Moderator
  • *****
  • Posts: 17816
  • Country: gb
  • Did that just blow up? No? might work after all !!
    • Simon's Electronics
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #16 on: September 05, 2021, 06:42:01 pm »
Well even if the oversampling does not work at 10 bit I have enough resolution. The 0 series mega's have hardware accumulation and the result ready interrupt fires after the 64 results are accumulated so it's painless to do, it sort of filters a bit and maybe gets me more resolution. I do not know if I have enough noise. I never bother to disable the digital buffers and given that the system powers a motor I suspect plenty of noise in the voltage an current readings.
 

Offline mariush

  • Super Contributor
  • ***
  • Posts: 5029
  • Country: ro
  • .
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #17 on: September 05, 2021, 06:48:42 pm »
Not sure why you'd use a voltage reference of 1117 mV when there's 2048mV (211) and 4096 mV (212) references and would work well with 12 bit or higher ADCs.

Those would allow you to just use some shift operations instead of multiplications or divisions.

May also want to consider lookup tables, for example may do the math with shifts instead of floating points and  have 4 bits or 8 bits per value as correction to the result you calculate. Maybe you can even figure out a set of values that can be reused (for adc readings 128 to 256, reuse lookup table for 0..127 and optionally add a fixed value)
 

Offline SimonTopic starter

  • Global Moderator
  • *****
  • Posts: 17816
  • Country: gb
  • Did that just blow up? No? might work after all !!
    • Simon's Electronics
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #18 on: September 05, 2021, 08:12:35 pm »
Not sure why you'd use a voltage reference of 1117 mV when there's 2048mV (211) and 4096 mV (212) references and would work well with 12 bit or higher ADCs.

Those would allow you to just use some shift operations instead of multiplications or divisions.

May also want to consider lookup tables, for example may do the math with shifts instead of floating points and  have 4 bits or 8 bits per value as correction to the result you calculate. Maybe you can even figure out a set of values that can be reused (for adc readings 128 to 256, reuse lookup table for 0..127 and optionally add a fixed value)


well it was meant to be 1.2V but it is actually 1.177 by the multimeter. I'll look into that one later, the 5V the resistor divider is supplied from is 4.995V, I guess it is the impedance of the Vref input that is pulling it down a bit. It's not the end of the world as long as it is consistent.

For the division by 8191 I will make it a division by 8192, or a 13 bit shift to the right having added 8192 to the number I am "dividing"

is your suggestion of a lookup table for the fractional parts that would be lost?
 

Offline SimonTopic starter

  • Global Moderator
  • *****
  • Posts: 17816
  • Country: gb
  • Did that just blow up? No? might work after all !!
    • Simon's Electronics
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #19 on: September 05, 2021, 08:18:00 pm »
And then there are non-linearities and likely the ADC in the microcontroller you are using isn't great to begin with.

https://ww1.microchip.com/downloads/en/appnotes/atmel-2559-characterization-and-calibration-of-the-adc-on-an-avr_applicationnote_avr120.pdf

By the sounds of it not worth worrying too much about non linearities. The offset and gain can be easily compensated for but after that not worth worrying. They seemed to avoid talking about temperature compensation.

It sounds like the opamps in the AVR used for differential measurements are shit which hardly surprises me.
 

Online T3sl4co1l

  • Super Contributor
  • ***
  • Posts: 21687
  • Country: us
  • Expert, Analog Electronics, PCB Layout, EMC
    • Seven Transistor Labs
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #20 on: September 05, 2021, 10:37:35 pm »
Why do the units matter at all?  The ADC just measures a ratio to VREF.  All the math inside can be dimensionless and you don't have to worry about scale until the very end when printing to the user.

Starting with a 10 or 12 bit ADC, you may have enough fractional capacity in a 16 bit variable to do useful work.  I did this on my reverb effect box (XMEGA based).  16 bits was a bit tight, making some overflow tricky to handle (implementing saturating arithmetic rather than wrapping overflow; I was also using signed arithmetic to simplify summing/filtering operations).  Some operations were done with a 24 or 32 bit accumulator; you won't gain anything in C using 24 bits (there is a (u)int24_t primitive available in avr-gcc, but it's basically cast to 32-bit for all arithmetic operations) so just stick to 32 when you need more.

And yes indeed, you can do fixed point by implicitly multiplying everything by some [bit] offset; be very careful to remove the excess bits when multiplying, and make sure arithmetic is done with enough bits to fit the result before shifting: to C, it is absolutely natural to multiply two 16-bit variables into a 16-bit result, as absurd as that is mathematically.  In particular, byte shifts are basically free (GCC is awful at optimizing them, but again, unless you're doing ASM, it's just what you have to live with).

So, for example the product of two 8.8 fixed point variables, as 8.8 result, must be written:
Code: [Select]
int16_t fracmul(int16_t a, int16_t b) {
return ((int32_t)a * b) >> 8;
}
but you should also check for overflow to implement saturating arithmetic.

Like uh, let me see,
Code: [Select]
int16_t fracmul(int16_t a, int16_t b) {
int32_t p = (int32_t)a * b;
if (p > 0x00ffffff) return 0x7fff;
if (p < -0x00ffffff) return 0x8000;
return p >> 8;
}
There are better ways to do it (the compiler may realize the low bytes don't need to be checked, but low optimization will do the full four-byte comparison verbatim) but this should at least be correct.  (I'm quite proud of the few-CPU-cycle solution I found, in my assembler code, but it's special-casey of course and not something you can do in GCC alone.)

Tim
Seven Transistor Labs, LLC
Electronic design, from concept to prototype.
Bringing a project to life?  Send me a message!
 

Online PlainName

  • Super Contributor
  • ***
  • Posts: 6845
  • Country: va
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #21 on: September 05, 2021, 11:15:18 pm »
Quote
surely the static one needs to be a global volatile? or it will just always be "0" having just been created.

A static variable in a function means it's on the heap rather than stack (so a 'global' that can only be seen by that function). The '0' is simply the initial value and is only set at program boot, not every time.
 

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 26907
  • Country: nl
    • NCT Developments
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #22 on: September 06, 2021, 12:00:29 am »
Why do the units matter at all?  The ADC just measures a ratio to VREF.  All the math inside can be dimensionless and you don't have to worry about scale until the very end when printing to the user.
That is horrible for diagnostics and debugging. It is extremely handy & efficient to have the ADC value in mV or some other unit you are measuring because you can relate the value (you get from debug output and/or by watching a variable through a debugger) 1 on 1 with what is applied to the ADC externally. Using meaningful units takes away a source of error and confusion in a situation where a circuit and/or piece of software isn't behaving as it should OR during verification.
« Last Edit: September 06, 2021, 12:02:12 am by nctnico »
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline David Hess

  • Super Contributor
  • ***
  • Posts: 16618
  • Country: us
  • DavidH
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #23 on: September 06, 2021, 02:51:49 am »
Why do the units matter at all?  The ADC just measures a ratio to VREF.  All the math inside can be dimensionless and you don't have to worry about scale until the very end when printing to the user.

That is horrible for diagnostics and debugging. It is extremely handy & efficient to have the ADC value in mV or some other unit you are measuring because you can relate the value (you get from debug output and/or by watching a variable through a debugger) 1 on 1 with what is applied to the ADC externally. Using meaningful units takes away a source of error and confusion in a situation where a circuit and/or piece of software isn't behaving as it should OR during verification.

I agree with T3sl4co1l.  The meaningful units are "counts".
 

Offline JPortici

  • Super Contributor
  • ***
  • Posts: 3461
  • Country: it
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #24 on: September 06, 2021, 04:17:04 am »
BTW, XC8 now includes avrgcc for the avr mcus, so this is 'gcc' we are talking about.

Edit:
Microchip use gcc for XC32, XC16 and XC8-AVR
Basic optimisations are free but they charge for higher level optimisation.
All source code is available so they comply with gpl license but the build environment is obviscated to hinder re-compilation.
For XC16 XC32 there is this medicine: https://github.com/cv007/XC3216
Not sure about XC8-AVR.

XC8-PIC uses LLVM I believe, so they can lock that down.

-O2 is not enough? Only -O3 -Os and their own things (functional safety certified compiler/code-stack analysis/profiling) are paid options
soft floating point should be already "optimized" anyway, if not the compiler writers should be informed.
 

Offline SimonTopic starter

  • Global Moderator
  • *****
  • Posts: 17816
  • Country: gb
  • Did that just blow up? No? might work after all !!
    • Simon's Electronics
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #25 on: September 06, 2021, 08:16:09 am »
I'm trying to get both meaningful numbers and retain the values. The final conversion to human numbers does not need to be too accurate. What is more important is being able to write a define in mV and use that as a threshold. Once I know the scalling factor I can scale it to match. In fact I dont even need to do the last division until I want human readable numbers but that does leave every result as a 32 bit variable and bit shifting is still probably faster than division.
 

Offline voltsandjolts

  • Supporter
  • ****
  • Posts: 2300
  • Country: gb
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #26 on: September 06, 2021, 09:03:32 am »
BTW, XC8 now includes avrgcc for the avr mcus, so this is 'gcc' we are talking about.

Edit:
Microchip use gcc for XC32, XC16 and XC8-AVR
Basic optimisations are free but they charge for higher level optimisation.
All source code is available so they comply with gpl license but the build environment is obviscated to hinder re-compilation.
For XC16 XC32 there is this medicine: https://github.com/cv007/XC3216
Not sure about XC8-AVR.

XC8-PIC uses LLVM I believe, so they can lock that down.

-O2 is not enough?...

For most people I am sure -O2 is enough optimisation. But that's not my main point.
Microchip take GPL software, modify 0.01% of it, then charge money to unlock features.
They have the right to do this as long as they publish source code to meet GPL requirements, which they do.
The GPL also gives me the right to unlock these features (IMO, IANAL) either by binary modification or other means, should I choose to do so.
 

Offline SimonTopic starter

  • Global Moderator
  • *****
  • Posts: 17816
  • Country: gb
  • Did that just blow up? No? might work after all !!
    • Simon's Electronics
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #27 on: September 06, 2021, 01:37:09 pm »
I tried to set avr-gcc up but it did not work. 
 

Offline voltsandjolts

  • Supporter
  • ****
  • Posts: 2300
  • Country: gb
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #28 on: September 06, 2021, 04:24:52 pm »
If you are using AVR and you are using XC8, then you are already using avrgcc. It's built into "XC8" which is now a combination of two different compilers (1) the same old XC8 for PIC based on LLVM and (2) avrgcc.
Have a look here and see what you find: C:\Program Files\Microchip\xc8\

If you aren't using that, what compiler are you using?
 

Offline SimonTopic starter

  • Global Moderator
  • *****
  • Posts: 17816
  • Country: gb
  • Did that just blow up? No? might work after all !!
    • Simon's Electronics
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #29 on: September 06, 2021, 06:50:38 pm »
Well I am using XC8, but I want to know even if it's running FCC that it is optimised and not crippled then wrapped up.
 

Offline voltsandjolts

  • Supporter
  • ****
  • Posts: 2300
  • Country: gb
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #30 on: September 06, 2021, 07:18:55 pm »
Optimisation settings available are here:
Project Properties > XC8 Global Options > XC8 Compiler > Option : Optimisations > Level [0,1,2 are free]  [3,s require the paid compiler]

If you are happy with level 0,1,2, then all is well. TBH its probably fine.

If it makes you gnash your teeth (like it does me somewhat) then this might be a good starting point for 'fixing' it:
https://github.com/cv007/XC3216/blob/master/xc8-avr-info.txt

From what you said above, sounds like just downloading avrgcc and asking MPLABX to use that doesn't work (no surprise there, thanks microchip).
« Last Edit: September 06, 2021, 08:05:26 pm by voltsandjolts »
 

Online Kleinstein

  • Super Contributor
  • ***
  • Posts: 14203
  • Country: de
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #31 on: September 06, 2021, 08:34:52 pm »
At least with GCC there is not much use of -O3. It sometimes produces very long code by loop unrolling and this is about the main point it does extra.
The normal choice is -Os, so optimization for code size.

For performance it really help to us the fixed point math with just shifts for the divider, and do the conversion to normal units only at the end. Much of the math use of the resuslts should be with just the result from the ADC with no scaling at all. Only the user may later care about units like mV.
If you need to compare to a fixed limit (e.g. check if voltage is < 0.5 V, than do the scaling on the limit side - chances are the compiler or preprocessor would to the math and not the µC.
 

Offline SimonTopic starter

  • Global Moderator
  • *****
  • Posts: 17816
  • Country: gb
  • Did that just blow up? No? might work after all !!
    • Simon's Electronics
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #32 on: September 07, 2021, 08:55:18 pm »

your teeth (like it does me somewhat) then this might be a good starting point for 'fixing' it:
https://github.com/cv007/XC3216/blob/master/xc8-avr-info.txt

From what you said above, sounds like just downloading avrgcc and asking MPLABX to use that doesn't work (no surprise there, thanks microchip).

Correct but if AS has been installed then it works so it's something to do with the setup and the MPLABX/Net beans settings do not tell all.
 

Offline voltsandjolts

  • Supporter
  • ****
  • Posts: 2300
  • Country: gb
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #33 on: September 08, 2021, 08:07:24 am »
I wouldn't be surprised if MPLABX is somehow tied to using XC8-AVR, such that you can't configure it to use a standard download of avr-gcc.
Such is the Microchip philosophy on compilers.

Edit:
I see that Atmel Studio (that Visual Studio based IDE) has now been rebranded as Microchip Studio.
The blurb says it now supports XC8...hmm, does that mean it doesn't support avr-gcc directly?

This might be a workaround to get a clean avr-gcc running in MPLABX (haven't tried it myself)
https://www.avrfreaks.net/forum/tutsoft-how-set-mplab-x-ide-use-different-compilers-windows-os

« Last Edit: September 08, 2021, 09:47:37 am by voltsandjolts »
 

Offline SimonTopic starter

  • Global Moderator
  • *****
  • Posts: 17816
  • Country: gb
  • Did that just blow up? No? might work after all !!
    • Simon's Electronics
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #34 on: September 08, 2021, 01:19:08 pm »
Microchip have a separate download for AVR-GCC, this is what I tried. I only use MPLABX as it will be available on linux if I make the switch and Atmel/Microchip Studio only works with admin rights which would be a nightmare at work. I suspect that at some point it will vanish as surely they have to pay microsoft some sort of bulk licence.
 

Offline snarkysparky

  • Frequent Contributor
  • **
  • Posts: 414
  • Country: us
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #35 on: September 08, 2021, 04:18:55 pm »
Old versions of Atmel Studio are available.   Made before Microchip changed the compiler.  Should be able to get full optimizations from them
 

Online PlainName

  • Super Contributor
  • ***
  • Posts: 6845
  • Country: va
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #36 on: September 08, 2021, 04:40:34 pm »
Quote
surely they have to pay microsoft some sort of bulk licence

I wouldn't put it past Microsoft to have given Microchip a backhander to get their IDE in place. Their (Microsoft's) plans would see any such license fee as not even rating 'dropped down the back of the sofa' compared to what they will reap once everyone is using VSCode and paying cloud fees to access their projects. And ongoing fees for access to user data.

They've already started down that route with Python.
 

Offline SimonTopic starter

  • Global Moderator
  • *****
  • Posts: 17816
  • Country: gb
  • Did that just blow up? No? might work after all !!
    • Simon's Electronics
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #37 on: September 08, 2021, 07:29:24 pm »
There is really nothing of VS available in AT/MS, at one point it was a way of getting a free licence until they locked it down in later versions so I think Microsoft do rather care or they would not have stopped giving away the whole thing.
 

Offline SimonTopic starter

  • Global Moderator
  • *****
  • Posts: 17816
  • Country: gb
  • Did that just blow up? No? might work after all !!
    • Simon's Electronics
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #38 on: September 20, 2021, 02:08:26 pm »
OK so I am trying to use floats. Now the trouble starts apparently. So i try sending 3.14 in float format to a serial port and read it on realterm.

I get 3. looking at the microchip developer help: https://microchipdeveloper.com/c:understanding-floating-point-representations
it turns out they had to do their own thing just to spice things up. Floats are 24 bit in XC8. realterm only has one option of float visualization and I think it's 32 bit as it's called float4.

So how do I make XC8 use 32 bits. is there a more user friendly serial terminal out there?
 

Online T3sl4co1l

  • Super Contributor
  • ***
  • Posts: 21687
  • Country: us
  • Expert, Analog Electronics, PCB Layout, EMC
    • Seven Transistor Labs
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #39 on: September 20, 2021, 02:55:52 pm »
What, did they not implement float printing for their custom data type?  So it's just implict casting to int or whatever?

Tim
Seven Transistor Labs, LLC
Electronic design, from concept to prototype.
Bringing a project to life?  Send me a message!
 

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 26907
  • Country: nl
    • NCT Developments
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #40 on: September 20, 2021, 03:09:54 pm »
Likely it needs linking a different library which has support for floating point.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline voltsandjolts

  • Supporter
  • ****
  • Posts: 2300
  • Country: gb
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #41 on: September 20, 2021, 03:30:48 pm »
The PICs have the option of 24 or 32 bit float, but AFAIK the AVRs only use 32bit.
The XC8 AVR manual is here.

Quote
4.3.4 Floating-Point Data Types
The MPLAB XC8 compiler supports 32-bit floating-point types. Floating point is implemented using a IEEE 754 32-bit
format. Tabulated below are the floating-point data types and their size.
 

Offline SimonTopic starter

  • Global Moderator
  • *****
  • Posts: 17816
  • Country: gb
  • Did that just blow up? No? might work after all !!
    • Simon's Electronics
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #42 on: September 20, 2021, 03:35:16 pm »
well I am confused then, if I make a
float test1 = 3.14 ;
and send it out of the serial port I get in hex 0x00 0x00 0x00 0x03
 

Offline voltsandjolts

  • Supporter
  • ****
  • Posts: 2300
  • Country: gb
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #43 on: September 20, 2021, 03:36:48 pm »
Sound like it's being cast to an integer. Post your transmit code.
 

Offline SimonTopic starter

  • Global Moderator
  • *****
  • Posts: 17816
  • Country: gb
  • Did that just blow up? No? might work after all !!
    • Simon's Electronics
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #44 on: September 20, 2021, 05:52:22 pm »
I don't have it to hand now but uh, yea, you are right.... I wrote a function that goes something like

USART_32bit_send(uint32_t data_dataword){

usart_transmitt_buffer = data_dataword >> 24;
usart_transmitt_buffer = data_dataword >> 16;
usart_transmitt_buffer = data_dataword >> 8;
usart_transmitt_buffer = data_dataword ;
}

I've actually given up on the idea of floats here. Given that I am anyway pushed into 32 bit variables and 32 bit multiplications seem to happen quickly enough I am keeping everything in a 32 bit space. I'm not doing my own fixed point representation either. Given that 32 bits have to hold the full calculation of:

real_value = (ADC_result * ADC_ref * itput_voltage_scaling) / ADC_top_count ;

and I am working with thresholds so the actual value is not too important so long as I can write my inputs "in english".

So:
real_value = (ADC_result * ADC_ref_mV * itput_voltage_scaling) ;
and
Threshold_mV = ADC_ref_mV  * ADC_top_count ;

So I change the division for multiplications that are going to be optimised out anyway.
 

Offline Jan Audio

  • Frequent Contributor
  • **
  • Posts: 820
  • Country: nl
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #45 on: September 22, 2021, 02:46:59 pm »
Just remember float is not accurate.
You getting integer from the ADC so you better keep it integer math.
 

Online Kleinstein

  • Super Contributor
  • ***
  • Posts: 14203
  • Country: de
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #46 on: September 22, 2021, 05:03:45 pm »
I don't have it to hand now but uh, yea, you are right.... I wrote a function that goes something like

USART_32bit_send(uint32_t data_dataword){

}

This gives you the case to an inut32 in the function call. To get the binaly data one would need to use something like a pointer, memcopy or a union.

Using integrers is likely easier, espeically if the µC has no FPU.

The usual way is to avoid magic numbers in the code itself and define the thresholds as constants at the top. The constant would than be in ADC counts or maybe multiples (like a factor of 256) if soemthing like averaging is used for the ADC. The math would be in the definietion of the constant, like
# define adc_scale_factor   (max_count / ref_mV)    //  e.g. 4096 counts / 2500 mV
#define limt    1200 * adc_scale_factor                // for a 1200 mV limit

Essentially all the math would be done by the compiler / preprocessor

 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14476
  • Country: fr
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #47 on: September 22, 2021, 05:28:18 pm »
Second the integer operations-only approach, of course. Especially on such a limited device as what the OP uses. Just including software FP support will take a significant fraction of the available code memory anyway. In particular if you not only use FP operations, but also FP support in the xxprintf() series of functions.


 

Online T3sl4co1l

  • Super Contributor
  • ***
  • Posts: 21687
  • Country: us
  • Expert, Analog Electronics, PCB Layout, EMC
    • Seven Transistor Labs
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #48 on: September 22, 2021, 05:32:58 pm »
FYI, last time I used float on AVR (GCC standard 32 bit floats) it only added about 2kB, surprisingly slim.  I forget which all operations that used; multiply, add and convert from/to int probably, maybe divide?

Tim
Seven Transistor Labs, LLC
Electronic design, from concept to prototype.
Bringing a project to life?  Send me a message!
 

Online Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6264
  • Country: fi
    • My home page and email address
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #49 on: September 22, 2021, 05:48:58 pm »
Let's recap.  (I promise, this will be useful!)

A linear mapping from X0..X1 to Y0..Y1 is
    y = Y0 + (x - X0) × (Y1 - Y0) / (X1 - X0)
and its inverse is
    x = X0 + (y - Y0) × (X1 - X0) / (Y1 - Y0)

Let's assume we limit x to X0..X1, inclusive, and y to Y0..Y1, inclusive; and that X0 < X1 and Y0 < Y1.  Then, we need one unsigned integer multiplication and one unsigned integer division with an unsigned integer type capable of describing values from zero to (Y1-Y0)×(X1-X0), inclusive.  (The final or initial addition is in the type of x or y.)

Let's consider a situation where x represents a 12-bit ADC reading (unsigned integer counts between 0 and 4095) and y represents voltage in tens of microvolts (unsigned integer count between 0 and 499878, inclusive, with 500000 = 5.0V).  In other words, X0=0, X1=4096, Y0=0, Y1=500000.  The product (Y1-Y0)×(X1-X0)=2048000000 < 2³¹, so we can use 32-bit integer math here.
Code: [Select]
static inline uint32_t  adc_to_internal_voltage_units(const uint16_t adc)
{
    return ((uint32_t)adc * UINT32_C(500000)) / 4096;
}

static inline uint32_t  internal_voltage_units_to_adc(const uint32_t units)
{
    return units * 4096 / UINT32_C(500000);
}

Because the next power of ten of 500000 is one million, which is less than 2²⁰, we only need 20-bit unsigned integer multiplication by ten, and a 20-bit unsigned integer comparison to and subtraction by 100000, to convert the internal voltage units to a string.  We can do the same thing by converting the internal units as an unsigned integer, then put a decimal point between the fifth and the sixth digit, too.  An example implementation:
Code: [Select]
static inline char *internal_voltage_units_to_string(uint32_t units)
{
    static char  buffer[7];  /* V.vvvv plus end-of-string terminator. */

    for (char i = 1; i < 6; i++) {
        char  digit = '0';

        while (units >= 100000) {
            units -= 100000;
            digit++;
        }

        buffer[i] = digit;

        if (i < 5)
           units *= 10;
    }

    buffer[0] = buffer[1];  /* Move units digit left of decimal point */
    buffer[1] = '.';  /* Decimal point */
    buffer[6] = '\0';  /* End-of-string mark */

    return (char *)buffer;
}
Note that 10000 internal units is 1.0 V.

Compiling the above functions (removing static inline) on AVR-GCC 5.4.0 with -O2 for ATmega32U4 generates 30, 32, and 150-byte functions, respectively; using -Os shrinks the last one to 90 bytes.  The first function uses __muluhisi3 for 32-bit unsigned integer multiplication, the second uses __udivmodsi4 for 32-bit unsigned integer division, and the third is self-contained using -O2.  The third uses __muluhisi3 if using -Os; the difference between -O2 and -Os here is whether the multiply-by-ten is implemented via a call to __muluhisi3 or inlined as bit shifts and additions (the difference being about 60 bytes of code).

This same logic can be expanded into any internal units.  If the string form is needed, it is useful to have the unsigned integer units have the same digits (i.e., some power of ten, positive or negative, of the human-useful value), because that makes the string conversion easy and cheap.  Otherwise, a second linear mapping is needed (to convert the internal units to a decimal representation); which itself is not that costly, code-wise, as you can see from above.

In particular, the fully variable (run-time calibratable) form,
Code: [Select]
#include <stdint.h>

extern const int16_t  y_0;
extern const uint16_t  y_delta;  /* ydelta = y1 - y0, ydelta > 0 */
extern const int32_t  x_0;
extern const uint32_t  x_delta;  /* xdelta = x1 - x0, xdelta > 0 */
typedef  uint64_t  xy_type;

int32_t  x(const int16_t y)
{
    return x_0 + ((xy_type)(y - y_0) * x_delta) / y_delta;
}

int16_t  y(const int32_t x)
{
    return y_0 + ((xy_type)(x - x_0) * y_delta) / x_delta;
}
generates 140-byte and 160-byte functions using the same settings as above (for both -O2 and -Os). x() contains one call to __muldi3 and one call to __udivdi3, and y() one call to __mulsidi3 and one call to __udivdi3, so I would not consider them "slow". Note that the xytype is an unsigned integer type that can describe values from 0 to xdelta*ydelta, inclusive; here, a 48-bit unsigned type would suffice.  With x_delta=500000 and y_delta=4096, uint32_t would suffice; that would shrink the functions a bit and use faster calls.
« Last Edit: September 22, 2021, 06:12:24 pm by Nominal Animal »
 
The following users thanked this post: DiTBho

Offline mikerj

  • Super Contributor
  • ***
  • Posts: 3240
  • Country: gb
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #50 on: September 24, 2021, 08:35:44 am »
I am working with the ADC on an AVR. I am trying to get a result in a number that makes sense so that defines can be written in say mV. So the easy sounding solution is to work in float arithmetic but my guess is that this will mean adding a library I may not have room for and lots of CPU cycles.

Alternatively I can work in scaled up 32 bit integers. To get the actual number in mV (or mA) bearing in mind any scaling due to say input resistor dividers I tried to do:

"ADC result" x "ADC reference voltage in mV" x "input stepdown ratio" / "ADC maximum count"

But this turns out inaccurate for low values due to the division. With 13 bits, a reference of 1177 and no hardware scaling anything less than a result of 7 will produce 0 and I will of course not be retaining my resolution as every 7 counts will produce just 1 mV/mA.


It's not "inaccurate", it's doing exactly what you have said you wanted i.e. a value with a resolution of 1mV.  A count of less than 7 is less than 1mV, so why would you expect a different result?

If your actual issue is that it's not rounding to the nearest mV value than this is easily fixed by adding half the denominator to the numerator e.g.


mV = ((ADC_result * ADC_Vref_mV * Input_div) + (2^13)/2) /  2^13
 

Offline SimonTopic starter

  • Global Moderator
  • *****
  • Posts: 17816
  • Country: gb
  • Did that just blow up? No? might work after all !!
    • Simon's Electronics
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #51 on: September 24, 2021, 11:46:34 am »
My issue was that I wanted a number expressed in mV not of mV
 

Offline Bassman59

  • Super Contributor
  • ***
  • Posts: 2501
  • Country: us
  • Yes, I do this for a living
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #52 on: September 24, 2021, 03:22:59 pm »
My issue was that I wanted a number expressed in mV not of mV

I realize that I'm a native New Jersey American English Speaker, and not a Native Ol' Blighty English Speaker, but ...

I don't understand the difference between "number expressed IN mV" and "number of mV."
 

Offline SimonTopic starter

  • Global Moderator
  • *****
  • Posts: 17816
  • Country: gb
  • Did that just blow up? No? might work after all !!
    • Simon's Electronics
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #53 on: September 24, 2021, 04:22:05 pm »
A number expressed in mV could well be 0.15 mV, as opposed to the number "of" mV like 1, 2, 3.

I did not want to make life hard and used "V" as then I have defines like 3.5 (V) but 3500 (mV) is fine, it keeps the numbers I write in whole numbers, but I may still want to know the fractional part of mV.
 

Offline Siwastaja

  • Super Contributor
  • ***
  • Posts: 8173
  • Country: fi
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #54 on: September 24, 2021, 04:27:58 pm »
Just keep increasing the number of digits (and as a result, number of bits in the storage and calculation) until you are happy with the resolution. For example, use microvolts internally. Even if you need to go to int64_t types for calculation, storage, or both, the result is still way more efficient than software floating point library.

Although, if the data source is say 12-bit ADC, it's very hard to justify going beyond 32-bit datatypes unless you want a nearly endless string of meaningless numbers to be displayed.
 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14476
  • Country: fr
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #55 on: September 24, 2021, 04:47:31 pm »
A number expressed in mV could well be 0.15 mV, as opposed to the number "of" mV like 1, 2, 3.

I did not want to make life hard and used "V" as then I have defines like 3.5 (V) but 3500 (mV) is fine, it keeps the numbers I write in whole numbers, but I may still want to know the fractional part of mV.

Then use whatever sub-unit that gives you the expected resolution.

For instance, if you need 2 decimals (which means, a resolution of 10 µV), then use integers that express the value in units of 10 µV. For 0.15 mV, that would be the number '15'. For say 10.55 mV, that would obviously be '1055'. End of the story.

If you need to display the values in mV, the routine is trivial. Just use the same routine as if you were displaying an integer, but insert a decimal dot between the fractional part and the integer part. The routine just needs to count the number of decimals.

 

Online T3sl4co1l

  • Super Contributor
  • ***
  • Posts: 21687
  • Country: us
  • Expert, Analog Electronics, PCB Layout, EMC
    • Seven Transistor Labs
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #56 on: September 24, 2021, 05:00:20 pm »
That is, after all, all that floating point does, it just tracks the decimal for you.  You have a number at some scale, and simply put the decimal where you need to when printing it out (or comparing to numbers at other scales, etc.).

Tim
Seven Transistor Labs, LLC
Electronic design, from concept to prototype.
Bringing a project to life?  Send me a message!
 
The following users thanked this post: Bassman59

Offline Siwastaja

  • Super Contributor
  • ***
  • Posts: 8173
  • Country: fi
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #57 on: September 25, 2021, 09:35:59 am »
And if you just feel lazy and non-pedantic-y...

Just use like microvolts, microamps and int64_t for everything, don't bother checking any ranges, and be done with it.

It's highly unlikely you overflow anything in practice, and you also have enough resolution for nearly all practical purposes.

Yet int64_t is far more efficient than software floating point. Bloat due to laziness, but not nearly as much as with float, which is the ultimate solution for the most lazy.

I can't say I recommend this, though. I recommend always checking ranges and making sure worst-case numbers fit in the datatypes, each and every time, because that saves time in the long run, and enables you to get rid of oversizing "just in case".
 

Online Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6264
  • Country: fi
    • My home page and email address
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #58 on: September 25, 2021, 09:06:26 pm »
If you need to display the values in mV, the routine is trivial. Just use the same routine as if you were displaying an integer, but insert a decimal dot between the fractional part and the integer part. The routine just needs to count the number of decimals.
Or use something like the internal_voltage_units_to_string() I showed in my post above.  All it needs is a comparison and subtraction (in a loop) by a constant (that matches 1.0 in the fixed point format), and multiplication by ten.  It's not the fastest way to do it, but on devices like AVRs, it is quite small and efficient overall.

In general, you'll want to represent real numbers X using an integer V and a fixed nonnegative integer power of ten P, via
    X = V × 10-P = V / 10P
so that the inverse is
    V = X × 10P
noting that since P is a constant, 10P is a constant as well.

(The conversion function, using the multiply-by-ten-then-subtract algorithm shown in my earlier post, must use an unsigned integer type that can describe 0 to 10×10P-1 = 10P+1-1, inclusive.)

Since the ADC typically produces values that are not directly a power of ten of the measured quantity, mapping the ADC reading X to suitable real V, linearly, as shown in my previous post, achieves this in an easily configured (even runtime-configurable) manner.  Then, the only place where you really see 10P is in the fixed-point-(or-internal)-to-string function.

If your ADC produces values 0 to 4096, and the measured quantity is 0.0000 V to 5.0000 V, map ADC 0..4096 to 0..50000, and treat the latter as P=4, i.e. print the value using five digits, zero-padded, and insert the decimal point just after the leftmost (most significant) digit.
If the measured quantity is 0.0 mV to 5000.0 mV, map ADC 0..4096 to 0..50000, and treat the latter as P=1 (10¹ = 10), i.e. decimal point just before the rightmost (least significant) digit.
« Last Edit: September 25, 2021, 09:08:44 pm by Nominal Animal »
 

Offline mikerj

  • Super Contributor
  • ***
  • Posts: 3240
  • Country: gb
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #59 on: September 26, 2021, 08:51:41 pm »
My issue was that I wanted a number expressed in mV not of mV

Adjust your scaling factor so you store the value in 0.125 mV units, which is just above the resolution that the raw ADC value has and also can be stored in just three bits.  When it comes to displaying the value, the factional part is in the lowest three bits, the integer part is in the upper bits. e.g.

Code: [Select]
printf("%u.%u\n", myval>>3, (myval & 0x0007) * 125);  /* assumes an unsigned value */

If you want something which is more convenient for display space such as 0.1mV units you can do that to, though extracting the integer and fractional parts for display will require either a division/subtraction operation  or a modulo operation.
 

Online Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6264
  • Country: fi
    • My home page and email address
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #60 on: September 27, 2021, 08:51:43 am »
If you want something which is more convenient for display space such as 0.1mV units you can do that to, though extracting the integer and fractional parts for display will require either a division/subtraction operation  or a modulo operation.
The division is by fractional one, i.e. by 8 (which optimizes to a bit shift); and for the fractional part, one can use a table lookup, too.

For 1/8, the fractions are 0.0, 0.125, 0.25, 0.375, 0.5, 0.625, 0.75, and 0.875.  Rounding to a single digit, one can use .0, 0.1, .3, .4, .5, .6, .8, and .9 .

The division cannot really be avoided (although can be calculated for a fixed divisor using a double-width multiplication); in general, this works for any kind of integer fractions (say, where 27 = 1.0, 54 = 2.0, and 1 = 1/27 ≃ 0.037).
 

Online Kleinstein

  • Super Contributor
  • ***
  • Posts: 14203
  • Country: de
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #61 on: September 27, 2021, 09:30:29 am »
For the output one can avoid the fractional part by using integers relative to the last digit and than add the decimal point in the output procedure (e.g. itoa variation).  The usual itoa will however include a division.

If really needed one can avoid the division in the itoa part by doing the conversion from the other end, though this is not common:
The digit steps are with a multiplication by 10 and than take away the top digit (e.g. upper 4 bits) for output. This naturally works for fractional number, but could also work with scaled integers. It needs a single long multiplication for the scale factor and no extra division.
Especially for a weak CPU with no HW divide this can be quite a bit faster (and shorter), especially if one has a general scale factor anyway.
 

Online Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6264
  • Country: fi
    • My home page and email address
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #62 on: September 27, 2021, 02:05:09 pm »
If really needed one can avoid the division in the itoa part by doing the conversion from the other end, though this is not common:
I showed this in #49 internal_voltage_units_to_string().  The only operations needed is an unsigned integer multiply by 10, and (repeated) comparison and subtraction to 100000 (which corresponds to 1.0 in the fixed-point units chosen in that post).  That can be adapted to any other fixed point format.
 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14476
  • Country: fr
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #63 on: September 27, 2021, 05:08:08 pm »
If really needed one can avoid the division in the itoa part by doing the conversion from the other end, though this is not common:
I showed this in #49 internal_voltage_units_to_string().  The only operations needed is an unsigned integer multiply by 10, and (repeated) comparison and subtraction to 100000 (which corresponds to 1.0 in the fixed-point units chosen in that post).  That can be adapted to any other fixed point format.

Yep.

Now while itoa()-like functions would usually mostly be used (or at least, should mostly be used for this) for display purposes, it's pretty rare that even a software-based division, handled by the compiler, would be a bottleneck, as display functions themselves are likely to consume a lot more time. Just a thought. As always, optimize when necessary, but don't overdo it.
 

Online Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6264
  • Country: fi
    • My home page and email address
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #64 on: September 27, 2021, 10:40:03 pm »
As always, optimize when necessary, but don't overdo it.
Very true; my goal for showing that implementation was to show how to do it lightweight on an AVR.  (I do realize it was a bit too deep to fit in the discussion; apologies to everyone!)

As discussed in the Constructing short C strings ... thread and elsewhere, there are lots of ways to do the conversion, some of them simple, some of them efficient, and many both (but exactly which ones depends on the exact hardware architecture).  Even readable pure-standard C implementations are fast enough if used only say few hundred times per second or less.

Let me retry my original attempt from a different tack.  Hopefully, this will be easier to understand for everyone than my previous descriptions.

Any real value V can be approximated using an integer X in fixed point format using a constant factor F,
    XV × F
    VX / F

For some use cases, you want a power of ten F, for other use cases a power of two F, and for others an arbitrary, possibly even rational or irrational non-integer F.
Power of ten F is nice for display purposes; conversion to string is fast, but arithmetic operations (except addition and subtraction) slower.
Power of two F is nice for computation; conversion to string is slower, but arithmetic fast: integer speed except for a binary right-shift after each integer arithmetic multiplication or division.
Non-integer F are rare, and mostly used in special cases.  (Even X≃V², V≃sqrt(X), is more common, for example in computations involving 2D (lattice) Euclidean distances.)

Simon wants X to be human-readable in mV, but with at least one digit of precision.  The obvious solution is to have X=1 represent V=0.1mV, i.e. F=10 [mV]⁻¹.  Then, 5.0 V = 5 000 mV corresponds to X=50 000.  An uint16_t type X can then represent voltages between 0 and 6.5535 mV = 6 553.5 mV; useful on 8-bit architectures like AVRs.

However, the ADC in a microcontroller provides a ratiometric value, usually with some power of two having a specific voltage, and zero at zero volts (although the ADC might have to be biased using e.g. an opamp circuit if the interesting voltage range does not include zero).  This we solve by linearly mapping the ADC range to the correct X range, using exact integer math (because both are integers); this is the rational F case, F=N/D where N is the range (number of integer values) of X, and D is the range (number of integer values) of ADC results.  We only need to be careful to use integer types that are large enough to hold the intermediate integer values.

That mapping operation requires one multiplication and one division, and possibly an addition and/or subtraction (when one or both ranges do not start at zero).  If the full range of a typical binary ADC is used, multiplication by F is an integer multiplication (by N) followed by a binary right shift (to implement the division by power-of-two D), and the entire operation is rather fast.  (Also note that for runtime calibration, calibrating the V range – assigning correct voltage values to ADC zero reading and to the value the ADC compares to –, while keeping the ADC range D a constant power of two, means this efficiency is retained! Unfortunately, in practical circuits, it means we need to extrapolate the voltage from an ADC reading, rather than just store the ADC reading for known voltages, which can reduce the accuracy of the calibration.)

Using F=10 (for mV units) or F=10000 (for V units) makes display fast an easy, and does not affect addition or subtraction, but it does add an integer multiply-by-F when dividing by a voltage V, and an integer divide-by-F when multiplying by a voltage V.  If such arithmetic operations are rare compared to displaying the values as strings, then a power-of-ten-F makes sense.  If voltages are often used in multiplication and/or division, then using a power of two F would be more efficient, although then X would be in units of (some power of two) millivolts or volts.  (For tenth of millivolt precision, F=2³=8 would work, although F=2⁴=16 would allow all fractional digits to appear in the value; these two bracketing the tenth-of-a-millivolt precision.  Is octal human-readable? With F=8, X=01 (decimal 1) would refer to V=0.125, X=010 (decimal 8) to V=1.0, and say X=035 (decimal 29) to V=3.0+5/8=3.625.  However, X=07231 (decimal 3737) would refer to V=7×64+2×8+3+1/8=467.125, so I don't think larger octal values are that human-readable.)
« Last Edit: September 27, 2021, 10:47:15 pm by Nominal Animal »
 

Online Kleinstein

  • Super Contributor
  • ***
  • Posts: 14203
  • Country: de
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #65 on: September 28, 2021, 08:08:49 am »
Even with a binary factor for the frational part one can still do an efficient conversion to a ASCII format for the output. One just has to treat the whole number and fractional part seprate. The whole number part can use the normal atoi() method. For the fractional part one can multiply be 10 and that way get the ist digit after the decimal point as the new whole number part. Repeat that for each digit. This way can be even more efficient than the normal atoi() way (least significant digit first) used for whole number, just starting at the other end (most significant digit first).

With a 8 bit CPU this makes absolute sense if the shift for fractional is by a 8 bits multiple.

For the given original problem one usually does not need to the math and decision based on a scale to calculate with whole physical units like mV. The more logical way is to do the math with hardware related units, LSB (ADC steps) and maybe fractions of that and only convert to pysical units for output and input. Constants in the code (or better as a define at the top) can still be written in a readable way with an extra scale factor.
 

Offline mikerj

  • Super Contributor
  • ***
  • Posts: 3240
  • Country: gb
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #66 on: October 03, 2021, 10:30:14 am »
If you want something which is more convenient for display space such as 0.1mV units you can do that to, though extracting the integer and fractional parts for display will require either a division/subtraction operation  or a modulo operation.
The division is by fractional one, i.e. by 8 (which optimizes to a bit shift); and for the fractional part, one can use a table lookup, too.

For 1/8, the fractions are 0.0, 0.125, 0.25, 0.375, 0.5, 0.625, 0.75, and 0.875.  Rounding to a single digit, one can use .0, 0.1, .3, .4, .5, .6, .8, and .9 .

The division cannot really be avoided (although can be calculated for a fixed divisor using a double-width multiplication); in general, this works for any kind of integer fractions (say, where 27 = 1.0, 54 = 2.0, and 1 = 1/27 ≃ 0.037).

Agreed. Many years ago when working primarily with PICs and AVRs I wrote a little app to produce C code for division/multiplication to a specified accuracy using only shifts and addition/subtraction, not sure what happened to it.  A little care is needed to ensure you don't introduce rounding errors by throwing away LSBs too early.
 

Offline David Hess

  • Super Contributor
  • ***
  • Posts: 16618
  • Country: us
  • DavidH
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #67 on: October 03, 2021, 08:56:35 pm »
Agreed. Many years ago when working primarily with PICs and AVRs I wrote a little app to produce C code for division/multiplication to a specified accuracy using only shifts and addition/subtraction, not sure what happened to it.  A little care is needed to ensure you don't introduce rounding errors by throwing away LSBs too early.

Long ago for PIC, I implemented the base-2 log and antilog routines from Knuth's book using only shifts and adds to do multiplies, divides, powers, and roots.  In practice I ended up leaving a lot of variables and constants in log form.

« Last Edit: October 03, 2021, 08:58:14 pm by David Hess »
 
The following users thanked this post: mikerj, Nominal Animal

Offline mikerj

  • Super Contributor
  • ***
  • Posts: 3240
  • Country: gb
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #68 on: October 05, 2021, 07:10:27 pm »
Agreed. Many years ago when working primarily with PICs and AVRs I wrote a little app to produce C code for division/multiplication to a specified accuracy using only shifts and addition/subtraction, not sure what happened to it.  A little care is needed to ensure you don't introduce rounding errors by throwing away LSBs too early.

Long ago for PIC, I implemented the base-2 log and antilog routines from Knuth's book using only shifts and adds to do multiplies, divides, powers, and roots.  In practice I ended up leaving a lot of variables and constants in log form.

I wrote fixed point log/antilog functions for work maybe 15 years or so back, though it does use a small table as well as shift/add.  Pretty essential on small 8 bit micros when you need control loops etc. to run at a decent rate, but on modern 32 bit micros with a barrel shifter it means you hardly have to even think about the overhead of log operations.

I miss the heyday of the PICList with Scott Dattalo et al coaxing amazing functionality out of a handful of instructions.
« Last Edit: October 05, 2021, 07:12:20 pm by mikerj »
 

Online Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6264
  • Country: fi
    • My home page and email address
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #69 on: October 05, 2021, 10:32:44 pm »
Background stuff:

It is interesting to consider how fixed point, floating point, and pure exponential form are related.

In fixed point, a real number r is approximated with an integer n via a scaling constant S:
    r ≃ nS

In floating point, a real number r is approximated with an integer mantissa m and an exponent p of basis B:
    r ≃ m×Bp
For nonzero finite numbers in IEEE-754 Binary32 ("float") and Binary64 ("double"), B = 2 and the most significant bit of m, always 1, is implicit (not stored in memory).

In pure exponential form, a (nonnegative) real number r is represented via v, its logarithm base B, typically in fixed point format as an integer n via a scaling constant S:
    v = logBr ≃ n S
    r = Bv ≃ BnS

In fixed point, addition and subtraction is trivial, but multiplication requires a division by S.  In floating point, none of the operations are really trivial, but there exist some really clever algorithms to reduce the number of sub-operations needed.  In pure exponential form, addition and subtraction requires at least an antilogarithm (exponentiation), but multiplication and division are trivial:
    r1×r2 ≃ B(n1+n2) S
    r1/r2 ≃ B(n1-n2) S

An additional form I've seen, is integer rational representation, using a numerator n and a denominator (or divisor) d:
    r ≃ n / d
The "annoying" feature of this is that one often needs to divide both n and d by their greatest common divisor (but note that GCD can be implemented as a binary algorithm that does one bit per iteration, and only does subtractions and binary shift right by one bit at a time).
This is useful when dealing with exact numbers (and no irrational constants), particularly when n and d are represented as arbitrary-size integers.

A somewhat related form is when doing interval arithmetic.  Instead of a single real r, we use two, rmin and rmax, to specify the interval.  Arithmetic operators (+, -, ×, /) take a bit of care to implement correctly for these.  When dealing with multiple variables and their error bars, this is surprisingly useful: you at once know the full range (interval) of results, taking into the various terms intervals into account.  You can extend this by adding a third real, say peak, or median, but you need to be careful in defining its behaviour wrt. the above operators.
Generalizing this moves you into statistics and probability theory, especially probability distributions and probability density functions.  You can treat each numerical value as a probability density function, highest at the most likely values, and do basic arithmetic on them as you would on reals, getting the probability density function of the result.  Very useful in some cases, but rare.  Often the distributions are described using basis functions: say, as a (sum of) Gaussian(s) approximating the actual distribution, parametrised by two reals each: µ (mean/median/mode) and σ (standard deviation or "width").
(For example, consider you have numerical distributions representing say K variables and constants.  Instead of evaluating your arithmetic function NK times (given N samples per variable or constant) to construct a histogram of the results (approximating the probability density function of the result), you only evaluate the function once, but using probability density functions to describe each variable and constant.)

I just love how these things build on top of each other, and connect with seemingly completely unrelated (but useful) methods and descriptions.  That's why I love to explore these things, even if I never memorize the details, only absorb the connections and key details –– which can be used as tools to solve all sorts of odd problems, even if one needs to look up the nitty gritty details.
 

Online T3sl4co1l

  • Super Contributor
  • ***
  • Posts: 21687
  • Country: us
  • Expert, Analog Electronics, PCB Layout, EMC
    • Seven Transistor Labs
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #70 on: October 06, 2021, 12:27:26 am »
And on a less related note, I find intervals very useful in analog design: match up the input and output voltage/current ranges, and that's more or less it.

Or more generally, in CS: if a function doesn't operate (nominally) on the full range of its parameters, then explicitly test the extrema, the edge conditions around / beyond where it's defined.  Not always easy or feasible (how do you define range on a C string?), but can be very illuminating when possible.  And it's more every year, it's feasible to exhaustively test ~40 bits of parameters these days, and more with statistical coverage or fuzzing.  If, of course, you can afford the time to do such tests :)

Tim
Seven Transistor Labs, LLC
Electronic design, from concept to prototype.
Bringing a project to life?  Send me a message!
 

Online Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6264
  • Country: fi
    • My home page and email address
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #71 on: October 06, 2021, 02:44:08 am »
Interval arithmetic could be useful in embedded implementations when evaluating multiple variables, and decisions like "still within operational range" or "still outside operational range", avoiding issues related to oscillation (similar to Schmitt triggers).  Say, you might have a large number of sensors, or many noisy sensor readings, and use confidence intervals (say using the interval as the range that contains 75% of the samples).

I can imagine it could be useful in e.g. agriculture, where soil moisture sensors often fail (not the electronics, the sensor itself), so you might make a system of many cheap sensors, maybe with a red/green LED on top to show whether the sensor is considered useful or not (when directed by the system), so bad ones are trivial to replace.
 

Online T3sl4co1l

  • Super Contributor
  • ***
  • Posts: 21687
  • Country: us
  • Expert, Analog Electronics, PCB Layout, EMC
    • Seven Transistor Labs
Re: doing math, scaled up integer or float? and pre-processor stuff
« Reply #72 on: October 06, 2021, 09:26:03 am »
Kind of the opposite concept of fuzzy logic, though you could just as well say rather than being fuzzy, it's just got straight sides between what counts as each condition (above/in/below range).

Hmmm, you could do that multivariate as well, not that it would be exactly trivial to conceptualize, or use embedded.  Just that it reminds me of the error matrix of like a Kalman filter.

I suppose building it from the statistics of an array of sensors, you get kind of both, a statistical confidence as well as a fuzzy logic.

Tim
Seven Transistor Labs, LLC
Electronic design, from concept to prototype.
Bringing a project to life?  Send me a message!
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf