Data logging and graphing data on a TFT display

[DVX]

My latest project as an amateur designer has a BME280 sensor generating temperature, humidity and air pressure data at set time periods. These three sets of data are saved in 3 x arrays over a 24 hour period with the sampling every 7-8 minutes producing around 195 data points over 24 hours. I am coding in C on a PIC32 and displaying on a 320 x 240 TFT, my attempt at a graph showing temp is attached. So far I have plotted simple y axis values to generate a line graph, see pic, the graph is updated with the latest temperature reading on the right hand side and the graph line then moves one sample point to the left and is redrawn, this part works OK. I use an equation relating to the temperature at two known levels and where they should be shown on the graph, all graphing points are calculated for each array temperature value and then displayed.

So on to my question and request for help, I would like to scale the graph and show more of the rises and falls in temperature without distorting the total graph line, or it's overall Y axis value. I have tried a number of ways to do this, latest is to take the average of all the temperature readings in the temperature array and use this as a base line to monitor rises and falls, then only for these changes add a scaling factor. So far my scaling factor is fixed at x 2, doing this displays more changes in the y axis but introduces a problem as the average also changes as new temperature readings are taken and therefore moves this average base line. The scaling needs to work for temps declining or rising over many hours, or both, but some how still showing the line in the correct position on the graph. I hope this makes some sense and any ideas / feedback are welcome. With no scaling the graph line is displayed OK but smaller changes in temperature are hard to see. As I plan to read outside temperatures the y axis is from -20 to + 50C as shown.

BTW I have a lot of variable values on the being shown on the RHS of display as an easy way to see what is happening with the code producing the graph line and scaling.

[senso]

Change the Y axis max and minimum values so you more or less zoom in on the Y axis, instead of having a -20 to 50 range and having all temps on the 22-23ºC range.

[DVX]

Hi Senso, this could work by measuring min and max values stored in the array and then plotting these on the graph showing max deviation. One problem could be the position of the min and max range inside the -20 to +50 scale as it will vary with recorded temperatures, so it could show a moving window within the -20 +50 range. My preference would be to show a scaled graph on the -20 to + 50 y axis if possible as this would, at a glance, be quick and easy to read.     

[@rt]

Can you not work out the scale for each display frame? That way it couldn’t change frequently.

Just opinion, but you could right justify the blue text, use less space between the black text, and have a lot more graph.
Especially the “90”. Most variable width fonts have fixed width numeric digits, so it should at least be easy to fix that to a column.

Is the graph background yellow because of the temperature?

[DVX]

Hi @rt, I could make the graph wider expanding the x-axis by moving or changing the y-axis markers as you suggest. The x-axis show the last 24 hours and shows a moving timeline moving backwards from the right hand side, which shows current time. So the graph shows current and temperature over last 24 hours on a rolling graph display. The frame for the temperature measurement is fixed over the last 24 hours, this data is stored and is available.  The yellow display is due to the compiler I am using having a limited range of colours and yellow for now shows good contrast, this is easily changed for the final release. Once the first temperature sensor value is available, this is copied through the array to generate and display an initial flat line, new values are then added to the right hand side.

I am not sure yet how to scale the logged temperature values. For example, the line might average around 22C and is flat ( not changing) for the first 0 to 4 hours, then it ramps up to 26C over the 4 to 8 hours, then between 8 -12 hours it ramps back down to 22C, then 12 to current time has no temperature change. To scale the values, I do not want to change the flat part of the line, only the changing values, so the first four hours there is no scaling. Then the for the 4-8 hours period this is scaled to magnify the change with higher y axis values being shown. I have calculated the average of all the values and used this as a baseline to compare new reading to, but the average changes as new readings show change. Another way is to detect change from say, the last 5 data points (if all flat) and then scale the latest reading. Problem with this is the temp might fluctuate +- 1 or 2C and these fluctuations do not need scaling. Maybe what I am after is some form of fuzzy logic?

     

[@rt]

I’m probably not understanding the question properly about the graph.

but even with 16 colours the line can have more meaning free of charge assuming this is a refrigeration monitor or something where
there’s a definite desired range.



I thought the background colour might be an indicator.

[DVX]

The project is in part a weather station to monitor outside temperature over the last 24 hour period, so there is no one part of the temperature graph that needs special emphasis. It just has to show the whole temperature range in the last 24 hours and scale the parts that have changed to make them easier to see. Sorry if I didn't explain this well.   

Using different colours on the graph line looks interesting, thanks for generating this.

My problem is how to scale only the changing temperature values and leave the rest of the graph line unchanged. Maybe I need to define first what is a constant temperature (or average constant temperature) and then also for a changing temperature.  See new pic to explain better, the black line is original non-scaled temperature values and the red line is the scaled version. When there is no change both lines show the same background temperature level.

[senso]

That is just changing the max and min Y axis values, why have it go up to 50 to display at most 30ºC, make it a live max/min, round to the nearest 5ºC top and bottom if you want some consistency.

If you want to scale, you have to scale the axis, unless you leave the axis labels as fiction and scale it in the plot..

[DVX]

That is just changing the max and min Y axis values, why have it go up to 50 to display at most 30ºC, make it a live max/min, round to the nearest 5ºC top and bottom if you want some consistency.

If you want to scale, you have to scale the axis, unless you leave the axis labels as fiction and scale it in the plot..

50C is for a friend in Greece who will also build one, certainly 50C doesn't apply to the UK. Dropping the scale labels would make life easier, so will try this and then scale all values from the min y-axis value.

[David Chamberlain]

Hi DVX here is how I would approach it. Take your raw readings square them (preserving the sign) and then normalize it all. You'll see all the normalized data is between 0 and 1 where you can simply multiply by your display area height to get the y plot points.

The square provides the exaggeration effect your after or you could modify with some fractional power instead. The graph shows the squared one (RED) and just the normalized raw data.

[edit] The above method will see your vertical scale (min & max) changing all the time and giving you weird values. It would be better to round the min & max values before performing the normalize function. So for example to fix the data between the nearest 10 degrees...
max = ceil(max / 10) * 10
min = floor(min / 10) * 10

[DVX]

Hi DVX here is how I would approach it. Take your raw readings square them (preserving the sign) and then normalize it all. You'll see all the normalized data is between 0 and 1 where you can simply multiply by your display area height to get the y plot points.

The square provides the exaggeration effect your after or you could modify with some fractional power instead. The graph shows the squared one (RED) and just the normalized raw data.

[edit] The above method will see your vertical scale (min & max) changing all the time and giving you weird values. It would be better to round the min & max values before performing the normalize function. So for example to fix the data between the nearest 10 degrees...
max = ceil(max / 10) * 10
min = floor(min / 10) * 10

Thanks David, modeling your formula's in Excel produces the results I am looking for. You might be able to give me some ideas for the best way to position or move the whole scaled graph up or down on the y axis. This could be used to reference the graph line to a fixed point or to move the graph line to show max scaled deviation. My first thoughts are to find a way to detect when the graph line is not changing, so when it is close to or is flat horizontally.  This flat part and assuming if there are two or more then the lowest one is used, could then allow the whole graph line to be placed referenced to a fixed y axis (might be 0C ) or to move the whole graph line up or down to keep the scaled graph min and max values within the y axis min and max values.     

[David Chamberlain]

So I tried this (image below) but to me I think the solution is iterative, perhaps you or another member here can solve it in one step.

First I defined two constants, the set temperature that is our bass line (10 degrees) in yellow and where on the display you want this line 'set point' in blue I've picked 50% of the display height.

Next when we normalize we need to center around this point to subtract the squared setpoint temp (100 in green). Do this for the actual squared setpoint value as well so we end up with a normalized setpoint.

Subtract this normalized setpoint from the setpoint we want (0.5-0.1=0.4) in red 'setpoint_norm' then all the values above that in the table are derived from the norm(s)- this setpoint_norm

This will first ensure all the values are centered around the temperature you want. You'll note that the highest temperature now goes above 1.0 and this is the problem.

The only way to fix that is to iterate the min max values. You want to raise max till the largest value is less then or equal to 1 and lower the min value until the lowest value is greater then or equal to 0. You could do it in even jumps like I was describing in my last post +5, +10, +15 and so on.


The other part of your question about finding what this set point should be is just finding the value in your raw data that is closest to zero that would be easiest.

I think I probably explained all that badly so let me know if anything is unclear.

[edit] Here is the pseudo code because I did feel my explanation above was not clear.

Code: [Select]

t_raw[] = [-10,-2,3,2,0,10,...] // input temps
n_out[] // output normalized to display height.
// ^2 below is our function to exaggerate the data.
s_temp = 10 ^ 2 // 10 degrees for our desired fixed point.
display_point = 0.5  // put 10 degree point at the 50% vertical on display.
s_min = min(t_raw) ^ 2
s_max = max(t_raw) ^ 2
LOOP:
    s_maxmin = s_max - s_min
    n_temp = display_point + (s_min / s_maxmin) // our setpoint offset.
    for each i in t_raw
        n_out[i] = ((t_raw[i] ^ 2 - s_temp - s_min)  / s_maxmin) + n_temp
    next
    if(max(n_out) > 1)
        s_max+= 1
        goto LOOP
    if(min(n_out) < 0)
        s_min-= 1
        goto LOOP