Author Topic: Sensor Smoothing (Microcontrollers)  (Read 5325 times)

0 Members and 1 Guest are viewing this topic.

Offline shatteredTopic starter

  • Newbie
  • Posts: 3
  • Country: us
Sensor Smoothing (Microcontrollers)
« on: August 14, 2015, 12:35:48 pm »
I am a programmer and have only been playing on the electronics side for a couple of years.   I am self taught on the electronics side.   I am starting to get more involved in micro-controllers and I have encountered this issue several times.  I have solved the problem but I am not sure it is the correct way though.

Sensor data being sent to a micro-controller usually provides 100s of samples per second.  One of my past projects was a lie dectector, it started off as a just measuring the resistance of the persons skin but evolved to using to a micro-controller based system.  It was based off the make magazine "Truth Meter" (http://makezine.com/projects/the-truth-meter-2).  This project helped me to understand filtering and op-amps on the hardware side (circuit).  I took the output from the Truth Meter into an Arduino for further processing.  I tried to output the data to a LCD as a graph but I could not find a good method to smooth out the data so I just used 5 LEDs to register the changes in the resistance of the person's skin.  Worked pretty good could have been better but you have to start somewhere.   Now the problem is coming up again in my current project.  I am working on a low power ATMEGA 328P wireless sensor.  End state is to build basic sensor and test it to make sure my design is working.  I  then plan on added one of Atmel CryptoAuthentication chips to digitally sign the packets to add another layer of security. Concept is only to accept packets from digitally signed sources.   I have been using the EEVBlog battery tutorials to help design the lower power circuit and I would not be where I am with out those tutorials. 

I have a pcb on the way that has just engough to run the ATMEGA 328-P off a 3V power source, LEDS, voltage regulator etc gone.  I not sure if I will use the Arduino bootloader or not yet.   I need to test the power consumption on the base components to determine the best way to power the circuit before I add the power circuit to the design.  Which leads me to my issue of smoothing again.  I was looking for a way to log voltage and current consumption and the cost of logging multi-meters are to expensive for hobby use.  So I am finishing my own data logging circuit using an Arduino and INA219 for current and voltage monitoring.  Graphing the raw data the graph is very rough.  The data from the INA219 has a range of 0.03 V on a voltage source from a 7805.  Current readings have a range of .02 mA.   

The method I am using now is collecting a 100 samples and finding the mean then reporting that value as the final result.  It takes the Arduino about 500ms to run through all the calculations and push out the results.  I delay the program for about 500ms to collect my readings once a second.  Am I on the right track or is there a better way?  As I move forward and start collecting more data with other sensors I am going to run into this issue again so I am looking the best path forward.   

I am slowly pulling way from Arduino bootloader and will end up working with the ATMEGA directly.  I have background in assembly programming and have taken a micro-controller class which taught me how to read the data sheets and work with registers and EEPROM.  I like working with the chips that way as I had more control and things were very fast.  I am slowly adding the hardware I need to program ATMEGA chips without the Arduino bootloader just not there yet.   The development board I had to use for the class is the size of a PC motherboard and is too big to work with for everyday use.  I am thinking on purchasing the AVR Dragon as my programmer  because of the debugging it provides.  I have just switched to Atmel Studio for my Arduino programming and love it.  The debugging I can do with VisualMicro is so helpful.  No more writing debugging code to find out how long a loop takes to run or what my variables are doing.   

Any guidance would be helpful - thank you in advance 
 

Offline mikerj

  • Super Contributor
  • ***
  • Posts: 3231
  • Country: gb
Re: Sensor Smoothing (Microcontrollers)
« Reply #1 on: August 14, 2015, 01:15:13 pm »
You are mostly on the right track.  Averaging many samples certainly helps to remove noise, but a proper low pass digital filter is likely to be more effective, and you won't need a big buffer to hold all those results.

Atmel have an App note and source code for implementing digital filters on an AVR, which should be a good starting point.
 

Offline Kleinstein

  • Super Contributor
  • ***
  • Posts: 14016
  • Country: de
Re: Sensor Smoothing (Microcontrollers)
« Reply #2 on: August 14, 2015, 02:48:29 pm »
Just averaging 100 ADC results does not need memory for all the results. You just calculate the sum as the data come in. One might want to check for the min and max value too, to detect short time overflow/underflow.  Unless power is very critical its often better to takte many reading and average. Unsing less than about 4 slow readings is usually not good, as this would need a rather good anti aliasing filter - possibly taking more power than 3 additional readings.

If possible its also a good idea to avarage readings over integer muliples of the line period. So at 50 Hz in europe, use an intervall of 20 ms or 40 ms, but not 30 ms. This might mean adjusting the number of readings to some funny number, but scaling usually is often needed anyway. Even if needed an addition multiplication is cheap.
Using more readings might give you a little more resulution due to oversampling if enough noise is present (or added). So with 16 reading one could gain another 2 bit of resolution (but hardly more accuracy), at least in theory. It's often worth to keep a few of the extra Bits, and not just devide directly by the number of reading.

The best way to measure the highly dynamik current consumpltion of an µC in a low power application, is to use some kind of transimpedance amplifier. This needs something like a 9 V supply to bringt somthing like 3.5 to 2.5 V to the  µC. The resitor convering the current to voltage is in parallel to a relatively large capacitor to avarage out spikes. Also the voltage can be higher (e.g. up 5 V or more if the supply is high enough) - so not much problems with µV voltages. This way it's possible to measure nA avarage currents even with peaks in the mA range. Still the source impedance seen by the µC can be in the 100 Ohms range, with possible extra filtering.
 

Offline f5r5e5d

  • Frequent Contributor
  • **
  • Posts: 349
Re: Sensor Smoothing (Microcontrollers)
« Reply #3 on: August 14, 2015, 03:35:41 pm »
a length 100 running average filter does need a 100 long buffer to be able to subtract off the 100th sample from the running sum accumulator as the next new sample is added

but since the weights are unity you only have to subtract the one being pushed out, add the new and scale the sum - you don't need to touch the 99 samples in the middle, just increment the pointer for the circular buffer

running average is an example of a Finite Impulse Response filter - in general FIR do multiply the individual buffered samples by different weights each sample time


Infinite Impulse Response filters don't need as deep storage, are more similar to simulating the differential equation of the analog filter but have a irregular structure, aren't accelerated as much by "DSP" hardware, instructions - but do need often unexpectedly longer wordlengths for the calculations, intermediates and accumulators
« Last Edit: August 14, 2015, 04:21:42 pm by f5r5e5d »
 

Offline shatteredTopic starter

  • Newbie
  • Posts: 3
  • Country: us
Re: Sensor Smoothing (Microcontrollers)
« Reply #4 on: August 15, 2015, 05:54:27 am »
Thank you for feedback.  I am doing my research now on the digital filter design.  This is new to me so it will take some time to figure out but so far my initial research the concept is not too hard to understand, terms are the hardest part so far.  I sure will be back with more questions as I move this topic.
 

Offline hendorog

  • Super Contributor
  • ***
  • Posts: 1617
  • Country: nz
Re: Sensor Smoothing (Microcontrollers)
« Reply #5 on: August 15, 2015, 06:32:05 am »
This is a how I always do low pass filtering, it avoids the hassle of big buffers moving averages and is really simple.

raw = <get raw value from sensor>
filtered = raw * 1/10 + previous * 9/10;
previous = filtered

You are adding one tenth of the new raw value to 9 tenths of the filtered value. That means that the new raw value doesn't affect the filtered value very much. The value '10' can be changed to a larger or smaller number to get better or worse filtering. Just beware of overflow.

No need to maintain 100 sample buffer etc that you would otherwise need for a moving average.
 

Offline rs20

  • Super Contributor
  • ***
  • Posts: 2317
  • Country: au
Re: Sensor Smoothing (Microcontrollers)
« Reply #6 on: August 15, 2015, 06:42:42 am »
This is a how I always do low pass filtering, it avoids the hassle of big buffers moving averages and is really simple.

raw = <get raw value from sensor>
filtered = raw * 1/10 + previous * 9/10;
previous = filtered

You are adding one tenth of the new raw value to 9 tenths of the filtered value. That means that the new raw value doesn't affect the filtered value very much. The value '10' can be changed to a larger or smaller number to get better or worse filtering. Just beware of overflow.

No need to maintain 100 sample buffer etc that you would otherwise need for a moving average.
This method is called Exponential smoothing, just to give it an actual name. The impulse response is an exponential decay curve. Powers of 2 are better since divisions by powers of 2 is much cheaper.
 

Offline hendorog

  • Super Contributor
  • ***
  • Posts: 1617
  • Country: nz
Re: Sensor Smoothing (Microcontrollers)
« Reply #7 on: August 15, 2015, 06:51:41 am »
Quote
This method is called Exponential smoothing

Thanks that makes it easier to describe :)

 

Offline shatteredTopic starter

  • Newbie
  • Posts: 3
  • Country: us
Re: Sensor Smoothing (Microcontrollers)
« Reply #8 on: August 15, 2015, 08:20:06 am »
All of this is starting to make sense now.  I can replicate everything but I do not understand it.  I have now figured out that this falls in Digital Signal Processing which helps me now to define everything.  I have almost no formal training in electronics and I have been learning to do things but not understanding why I am doing them.  I am working on changing this by asking questions getting responses then spending hours trying to figure out why it was done that way.  Everyone's posts give me another piece of the pie. 

This forum is outstanding, other forums I get 9 troll answers and one solid answer.  I thank everyone for being so helpful and not commenting on my lack of knowledge in this area but instead taking their time to try to help me understand. 

Note:  Anyone else looking to understand this area better I found this reference:  http://www.dspguide.com/pdfbook.htm  It is taking the concepts mentioned here in the forum and explains them.  I never would have found this with out the responses here because I no idea what any of this was called. 

Tomorrow I plan trying this techniques out on some sample data. 
« Last Edit: August 15, 2015, 08:22:43 am by shattered »
 

Offline miceuz

  • Frequent Contributor
  • **
  • Posts: 387
  • Country: lt
    • chirp - a soil moisture meter / plant watering alarm
Re: Sensor Smoothing (Microcontrollers)
« Reply #9 on: August 15, 2015, 11:47:57 am »
Take a look at this post:

[url=http://electronics.stackexchange.com/questions/30370/fast-and-memory-efficient-moving-average-calculation/30384#30384]http://electronics.stackexchange.com/questions/30370/fast-and-memory-efficient-moving-average-calculation/30384#30384
[/url] -- it describes a simple and cheap IIR filter

I've rewritten filter analysis tools described in this post in Python to make it more accessible https://github.com/Miceuz/plotfilt

Also I've written a C++ class that's usable in Arduino environment that allows to cascade several IIR filters to create a higher order filter:

Code: [Select]

#define MAX_CASCADE_SIZE 5
typedef struct Filter {
    long filteredVal;
    unsigned char heaviness;
} Filter_t;

class FilterCascade {
public:
FilterCascade(unsigned char size, unsigned char heaviness[]);
long apply(long value);
private:
unsigned char size;
struct Filter filters[MAX_CASCADE_SIZE];
long filter(struct Filter *filter, long value);
};

FilterCascade::FilterCascade(unsigned char _size, unsigned char heaviness[]):
  size(_size)
{
  unsigned char i;
  if(size > MAX_CASCADE_SIZE) {
    size = MAX_CASCADE_SIZE;
  }

  for(i = 0; i < size; i++) {
    filters[i].heaviness = heaviness[i];
  }
}

long FilterCascade::filter(struct Filter *filter, long value) {
    filter->filteredVal = filter->filteredVal + ((value - filter->filteredVal) >> filter->heaviness);
    return filter->filteredVal;
}

long FilterCascade::apply(long value) {
    long filtered = value;
    int i = 0;
    for(i = 0; i < size; i++) {
        filtered = filter(&filters[i], filtered);
    }
    return filtered;
}

To use it, you go like this:

Code: [Select]

unsigned char heaviness[] = {2, 2};
FilterCascade filterCascade(2, heaviness);

//...
value = sensorRead();
valueFiltered = filterCascade.apply(value);


Offline ez24

  • Super Contributor
  • ***
  • Posts: 3082
  • Country: us
  • L.D.A.
Re: Sensor Smoothing (Microcontrollers)
« Reply #10 on: August 18, 2015, 06:51:06 am »
Quote
Note:  Anyone else looking to understand this area better I found this reference:  http://www.dspguide.com/pdfbook.htm  It is taking the concepts mentioned here in the forum and explains them.  I never would have found this with out the responses here because I no idea what any of this was called. 

thanks for the link to the pdf - I added it to (just in case it was not there):

https://www.eevblog.com/forum/beginners/electronics-primers-course-material-and-books/
YouTube and Website Electronic Resources ------>  https://www.eevblog.com/forum/other-blog-specific/a/msg1341166/#msg1341166
 

Offline hendorog

  • Super Contributor
  • ***
  • Posts: 1617
  • Country: nz
Re: Sensor Smoothing (Microcontrollers)
« Reply #11 on: August 18, 2015, 07:24:30 am »
A cool way to play with DSP filters cheaply is to grab a PSoc 5LP dev board for $10 USD:

http://www.cypress.com/documentation/development-kitsboards/cy8ckit-059-psoc-5lp-prototyping-kit

The dev tool which you can download for free (Windows only tho) has a slick filter designer.
http://www.cypress.com/documentation/component-datasheets/filter

There is also a sample project which connects the ADC -> Filter -> DAC
Its not a big learning curve from an Arduino as the IDE is very good. Its much more powerful though.


 

Offline Psi

  • Super Contributor
  • ***
  • Posts: 9875
  • Country: nz
Re: Sensor Smoothing (Microcontrollers)
« Reply #12 on: August 18, 2015, 07:26:34 am »
Exponential smoothing with shift ops is really fast.

Eg
ADCfiltered = ((ADCread() + (ADCfiltered*7)) >>3)

Adds 1 part new value and 7 parts avg value then divides by 8
You just have to check your ADCfilter variable can handle the *7 without overflow, or cast it

You could swap the *7 with a 3 left shift and then subtract itself once. Not sure if that'd be any faster but might be worth trying

ADCfiltered = ((ADCread() + ((ADCfiltered<<3)- ADCfiltered)) >>3)
« Last Edit: August 18, 2015, 07:33:36 am by Psi »
Greek letter 'Psi' (not Pounds per Square Inch)
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf