Author Topic: Conversion of ADC data to Q15/Q31 format  (Read 9304 times)

0 Members and 1 Guest are viewing this topic.

Offline syntax333Topic starter

  • Regular Contributor
  • *
  • Posts: 158
  • Country: 00
Conversion of ADC data to Q15/Q31 format
« on: January 15, 2020, 05:21:06 pm »
Hello, I am working on a DSP project, but I have never worked on such projects before so I am a newbie in this area.
Using the microprocessor PIC32MX795, I need to calculate the FFT of the data I read from the ADC.

I can read the data from the ADC, but it says in the library manual that I need to convert this ADC data to the Q15 format in order to use the functions available in the DSP library of the PIC32MX795.
Because the FFT function is expecting for the Q15 or Q31 format as input.

When I researched the Q15 format on the internet, I found that it would be enough to shift the integers I had read from ADC by 1 bit to the right.
For example:

ADC_Data = 3086 = 0b110000001110

shifting 1 bit to right 0b110000001110 becomes 0b0110000001110

Is that the case? Or is there anything else I have to do?

As I said I am new to this area so any information would be appreciated. Thanks in advance.
 

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 7198
  • Country: fi
    • My home page and email address
Re: Conversion of ADC data to Q15/Q31 format
« Reply #1 on: January 15, 2020, 05:59:25 pm »
Signed Q15 is a fixed point format where you have 15 fractional bits, and can represent values from -1 to 32767/32768≃0.9999695.
Logical value v corresponds to 16-bit signed integer ⌈32768×v⌋, and integer i corresponds to logical value i/32768.

Signed Q31 is a fixed point format where you have 31 fractional bits, and can represent values from -1 to 2147483647/2147483648≃0.99999999953.
Logical value v corresponds to 32-bit signed integer ⌈2147483648×v⌋, and integer i corresponds to logical value i/2147483648.

You do not necessarily need to do anything.  It depends on what the ADC readings represent.

The PIC32MX795 has a 10-bit ADC, which supports various output modes (page 239 of the datasheet, bits 8 to 10 of the AD1CON1 register).
The signed fractional 32-bit format (0b101) is compatible with Q31, and the (low 16 bits of the) signed fractional 16-bit format (0b011) is compatible with Q15.

That way the logical value that you operate the FFT on is between -1 and +1 (excluding the upper limit +1), -1 corresponding to the minimum voltage the ADC can measure, and the largest positive value (0.9999695) the maximum voltage the ADC can measure.

If you are only interested in some smaller range of values, and want to map them to -1..+1 or some subrange, then you do need to do some math conversion.  I can show you the exact math you need, although it really is just mapping the source integer values to target integer values with proper rounding, nothing special.
 

Offline syntax333Topic starter

  • Regular Contributor
  • *
  • Posts: 158
  • Country: 00
Re: Conversion of ADC data to Q15/Q31 format
« Reply #2 on: January 15, 2020, 06:35:49 pm »
Thank you for your reply.

It would be great if you can show me the math I need to convert the integer values to Q15.


Also from your explanation, can I assume that the example I have given is incorrect? (With bit shifting like : Value = (Value >> 1) & 0xFFFE))
« Last Edit: January 15, 2020, 06:43:07 pm by syntax333 »
 

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 7198
  • Country: fi
    • My home page and email address
Re: Conversion of ADC data to Q15/Q31 format
« Reply #3 on: January 15, 2020, 08:32:57 pm »
It would be great if you can show me the math I need to convert the integer values to Q15.
Like I said, if you use the signed fractional 16-bit format, the 16-bit ADC samples are in a suitable format.

If you want to convert values 0 to 1023, inclusive, to correspond to 0.0 to 0.9990234375=1023/1024 in Q15, just shift the value left by 6 5 bits.

Also from your explanation, can I assume that the example I have given is incorrect?
Yes.

Let's start from the beginning, then.

The ADC in your microcontroller has 10 bits of precision.  It does not just yield values 0 to 210-1, but has 8 different modes in which the results can be obtained:
  • Mode 0 = 0b000: Unsigned 16-bit integer.  The 32-bit ADC result words (ADC1BUFx) reads in binary 0000 0000 0000 0000 0000 00dd dddd dddd, i.e. between 0 and 1023, inclusive.
  • Mode 1 = 0b001: Signed 16-bit integer.  The 32-bit ADC result words (ADC1BUFx) reads in binary 0000 0000 0000 0000 0000 sssd dddd dddd, i.e. between 0 and 511, or 57344 and 65535, inclusive.  As a signed 16-bit integer, the value is between -512 and 511, inclusive.
  • Mode 2 = 0b010: Unsigned 16-bit fractional.  The 32-bit ADC result words (ADC1BUFx) reads in binary 0000 0000 0000 0000 dddd dddd dd00 0000, i.e. between 0 and 65472, inclusive, in steps of 64.
  • Mode 3 = 0b011: Signed 16-bit fractional.  The 32-bit ADC result words (ADC1BUFx) reads in binary 0000 0000 0000 0000 sddd dddd dd00 0000, i.e. between 0 and 32704, or 32768 and 65472, inclusive, in steps of 64. As a signed 16-bit integer, the value is between -32768 and 32704, inclusive, in steps of 64.
  • Mode 4 = 0b100: Unsigned 32-bit integer.  The 32-bit ADC result words (ADC1BUFx) reads in binary 0000 0000 0000 0000 0000 00dd dddd dddd, i.e. between 0 and 1023, inclusive.
  • Mode 5 = 0b101: Signed 32-bit integer.  The 32-bit ADC result words (ADC1BUFx) reads in binary ssss ssss ssss ssss ssss sssd dddd dddd, i.e. between 0 and 1023, inclusive.
  • Mode 6 = 0b110: Unsigned 32-bit fractional.  The 32-bit ADC result words (ADC1BUFx) reads in binary dddd dddd dd00 0000 0000 0000 0000 0000, i.e. between 0 and 4290772992, inclusive, in steps of 4194304.
  • Mode 7 = 0b111: Signed 32-bit fractional.  The 32-bit ADC result words (ADC1BUFx) reads in binary sddd dddd dd00 0000 0000 0000 0000 0000, i.e. between -2147483648 and 2143289344, inclusive, in steps of 4194304.
Above, d corresponds to value bits and s is sign bit.
Of these modes, 3 is compatible with Q15, and 7 is compatible with Q31.  So, if you use these modes, your ADC data is in suitable form for the FFT.



Fixed point integers are perfectly normal integers, that represent a different value.  For example, if x = 4251 = 0b0001000010011011, as a Q15 it represents value 4251/32768 ≃ 0.12973.

As the Fixed-point aritmetic article at Wikipedia explains, addition and subtraction of fixed-point integer types is done exactly like for normal integers, but multiplication and division involves an extra scaling (bit shift) operation.

The FFT library you have only supports logical values from -1 to just under +1, represented by fixed-point integers.

It is up to you to decide how you map your ADC readings for the FFT, but if you use ADC mode 3 or 7, it provides the 16- or 32-bit ADC samples in a form that corresponds to -1 (smallest or most negative voltage the ADC can detect) to +1 (greatest or most positive voltage the ADC can detect) in Q15 or Q31 fixed-point format.

You only need to do math on the ADC readings if you need a custom range, so that e.g. an specific ADC reading corresponds to 0, and another to (just under) +1.



Mathematically, scaling \$x\$ from \$[x_{min}, x_{max}]\$ to \$[y_{min}, y_{max}]\$ is
$$y = (x - x_{min}) \frac{y_{max} - y_{min}}{x_{max} - x_{min}} + y_{min}$$
This only applies to integers if we do rounding correctly.  Unfortunately, in C/C++, integer division truncates, i.e. rounds towards zero.  Most people, including Arduino developers, do not care about such minor details, so their map() function behaves differently for integer and floating-point values.

If we want to do rounding correctly, we need to use
$$y = \frac{(x - x_{min})(y_{max} - y_{min}) + r}{x_{max} - x_{min}} + y_{min}$$
where \$r\$ has magnitude just less than half of \$x_{max} - x_{min}\$, but has the same sign as \$y_{max} - y_{min}\$.  This yields the same as calculating the value using floating-point numbers using the mathematically correct formula, and then rounding it to an integer.  When programming, one needs to notice that the product is calculated first, and therefore the temporary variables need to be large enough (int32_t if using int16_t, for example).

For example, let's say we use ADC mode 0, but for say electrical reasons our readings are always between say 113 and 742, inclusive.
For the FFT, we wish to map these to say -0.25 to just under 0.75, which in Q15 correspond to integers -8192 and 24575.

Thus, we have \$x_{min} = 113\$, \$x_{max} = 742\$, \$y_{min} = -8192\$, \$y_{max} = 24575\$, and \$r = 314\$.  To convert a buffer full of samples we could use
Code: [Select]
/* Maps buffer values 113..742, inclusive, to -8192..24575, with correct rounding. */
void prepare_adc(int16_t buffer[], uint32_t count)
{
    for (uint32_t  i = 0; i < count; i++)
        if (buffer[i] <= 113)
            buffer[i] = -8192;
        else
        if (buffer[i] < 742)
            buffer[i] = ((int32_t)(buffer[i] - 113)*32767 + 314) / 629 - 8192;
        else
            buffer[i] = 24575;
}
which produces the same results if you used floating-point calculation,
Code: [Select]
            buffer[i] = round( (buffer[i] - 113.0) * 32767.0 / 629.0 - 8192.0);

Mathematically, \$r\$ is the additive rounding factor needed to round towards nearest integer instead of towards zero; and the integer division (by 629, above) in C and C++ rounds towards zero.
« Last Edit: January 16, 2020, 06:57:18 pm by Nominal Animal »
 

Offline syntax333Topic starter

  • Regular Contributor
  • *
  • Posts: 158
  • Country: 00
Re: Conversion of ADC data to Q15/Q31 format
« Reply #4 on: January 16, 2020, 03:20:35 pm »
Wow! Thank you very much for your detailed explanation.
Just to make sure that I get it right, if I wanted my 16-bit ADC data [0,4095] to be mapped from 0 to 0.9990234375(in this case 0x7FFF = 32767) my parameters becomes:
xmin=0 , xmax=4095, ymin=0, ymax=32767, and r=2047 right? After the mapping I just need to give the mapped array to FFT function?



If you want to convert values 0 to 1023, inclusive, to correspond to 0.0 to 0.9990234375=1023/1024 in Q15, just shift the value left by 6 bits.


While researching I just came a cross a youtube video by Eli Hughes which explains the topic of fixed point math. So correct me if I am wrong, Q15 format contains 1 sign bit right? If I want to convert 10-bit value to represent Q15, don't I need to shift the 10 bit data(0 to 1024) to left by 5 bits?
For example:

10bit data : bb bbbb bbbb
shifting left by 5 bits
bbb bbbb bbb0 0000

isn't there a hidden 0 at postion 16 which represents sign? so my Q15 format becomes

0bbb bbbb bbb0 0000
=
sbbb bbbb bbb0 0000

and my limits becomes 0 to 32736.

or is the sign bit extra to 16 bit data?

Do I lose 1 bit to sign information if I used "Mode 3 = 0b011: Signed 16-bit fractional."?

« Last Edit: January 16, 2020, 06:06:16 pm by syntax333 »
 

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 7198
  • Country: fi
    • My home page and email address
Re: Conversion of ADC data to Q15/Q31 format
« Reply #5 on: January 16, 2020, 07:31:15 pm »
Wow! Thank you very much for your detailed explanation.
Just to make sure that I get it right, if I wanted my 16-bit ADC data [0,4095] to be mapped from 0 to 0.9990234375(in this case 0x7FFF = 32767) my parameters becomes:
xmin=0 , xmax=4095, ymin=0, ymax=32767, and r=2047 right? After the mapping I just need to give the mapped array to FFT function?
Yes; except the range is 0.0 to 0.9999695 (≃ymax/32768).

If range 0.0 to 0.99975586 is okay, i.e. 0 to 32760/32768 inclusive, and you don't mind a fractional shift, then you can just shift each sample left by three bits.  In that case, ADC 0 would be mapped to 0.0 in Q15, and ADC 4096 to 1.0 in Q15 (except that 4095 is the biggest value).
(This corresponds to xmin=0, xmax=4096, ymin=0, ymax=32768, r=0, but is much faster to calculate.  The rounding just shifts the entire scale half an integer up.)

If you want 0.0 to 0.9990234375 (=32736/32768), then use xmin=0, xmax=0, ymin=0, ymax=32736, r=2047.

While researching I just came a cross a youtube video by Eli Hughes which explains the topic of fixed point math. So correct me if I am wrong, Q15 format contains 1 sign bit right? If I want to convert 10-bit value to represent Q15, don't I need to shift the 10 bit data(0 to 1024) to left by 5 bits?
For example:

10bit data : bb bbbb bbbb
shifting left by 5 bits
bbb bbbb bbb0 0000

isn't there a hidden 0 at postion 16 which represents sign? so my Q15 format becomes

0bbb bbbb bbb0 0000
=
sbbb bbbb bbb0 0000

and my limits becomes 0 to 32736.
Yes, that is correct.  I had a typo in my answer, fixed now.

Do I lose 1 bit to sign information if I used "Mode 3 = 0b011: Signed 16-bit fractional."?
It depends on which kind of ADC sampling mode you use.  Your microcontroller supports differential readings, so I believe the signed fractional range is intended for those, because then the ADC can return negative values too.

For unipolar measurements, I think you would lose one bit using mode 3, but I could be wrong.  (I use AVRs and ARMs myself, and haven't used PICs.)

You cannot use mode 2 = 0b010 for unipolar measurements, because that would map upper half of the ADC values to negative Q15 values.

If you use Mode 0 = 0b000, unsigned 16-bit, then shifting each sample left by 5 bits is a sensible option.  Then, the maximum ADC reading corresponds to 1023/1024 = 32736/32768 = 0.9990234375 for the FFT.

If you use Mode 0, but need the ADC values to span 0.0 to 0.9999695 for the FFT, use (x*32767+511)/1023 (which is equivalent to the precise formula using xmin=0, xmax=1023, ymin=0, ymax=32767, r=511).

That division will be slow, but it just so happens that if we use (x*32799+527)>>10 we get the exact same results for x=0..1023, but with the cost of one multiplication, one addition, and one bit shift right.  Your microcontroller has a single-cycle 32×16 multiple-add, so this will be very fast.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf