Live plots with gnuplot

[RoGeorge]

Trying gnuplot for the first time was fun. 
Though, it escalated fast. 

For example, one can take many measurements in a text file, a value on each line, then plot the measurements.txt without fiddling with programming, and without importing it in a spreadsheet.  It can be as easy as this:

Code: [Select]

gnuplot -p -e 'plot "measurements.txt"'
First thought was to use it for plotting live data from instruments, or from sensors.  gnuplot shines with static charts, but it is not that good at plotting animated/live charts.  Eventually gnuplot can pipe in the data to be plotted, but that is more of a hack than normal POSIX pipe. 

I've tried to live plot the CPU temperature, but got sporadic errors when the chart is updated faster than 10 times a second or so.  The errors may vary, most often it's just a warning:

"settings.gp" line 4402: warning: Skipping data file with no valid points

some other errors examples (much rare):

Qt: Session management error: networkIdsList argument is NULL
"settings.gp" line 102394: warning: Skipping data file with no valid points
cat: /dev/shm/cpu_temp_all.txt: No such file or directory
"settings.gp" line 104678: warning: Cannot find or open file "/dev/shm/cpu_temp.txt"
"settings.gp" line 104678: No data in plot

To plot the live temperature, I've used 2 scripts, a bash script 'tempCPUtoRAM.sh' that periodically reads the CPU temperature from '/sys/class/thermal/thermal_zone0/temp' then tail the last values to a file 'cpu_temp.txt', file that will be used by gnuplot to plot a live rolling chart of the CPU temperature, and another gnuplot script 'settings.gp' that keeps rereading (replotting) the data.

My 'tempCPUtoRAM.sh' bash script:

Code: [Select]

#!/bin/bash

# USAGE: 'tempCPUtoFILE <lastN> <delayS> <tmpPATH>'
#         tempCPUtoFILE 600 0.2 /tmp    # 600 samples taken at 0.2s written in /tmp
#         tempCPUtoFILE 600 0.2         # 600 samples taken at 0.2 swritten in /dev/shm
#         tempCPUtoFILE 600             # 600 samples taken at 1s written in /dev/shm
#         tempCPUtoFILE                 # 60 samples taken at 1s written in /dev/shm

# /dev/shm is always a RAM disk, used here as default path
#    manually change it to /tmp if shared memory disk not available

#VARIABLE=${1:-DEFAULTVALUE}
LASTN=${1:-60}
DELAY=${2:-1}
PLACE=${3:-/dev/shm}
echo cpu_temp_all.txt and cpu_temp.txt temporary saved in:  $PLACE

# set a trap at exit, to delete the created work files, e.g. after CTRL+C or KILL
trap 'rm $PLACE/cpu_temp_all.txt; rm $PLACE/cpu_temp.txt; \
      echo -e "\n$PLACE/cpu_temp_all.txt and $PLACE/cpu_temp.txt deleted"' EXIT

# keep <lastN> CPU temperature samples in <tmpPATH>/cpu_temp.txt
touch $PLACE/cpu_temp_all.txt > $PLACE/cpu_temp_all.txt
while true
    do
        cat /sys/class/thermal/thermal_zone0/temp >> /dev/shm/cpu_temp_all.txt && \
        cat $PLACE/cpu_temp_all.txt | tail -n $LASTN > $PLACE/cpu_temp.txt && \
        sleep $DELAY
    done

My 'settings.gp' gnuplot script':

Code: [Select]

plot "/dev/shm/cpu_temp.txt" u ($1/1000.0) w l lc 'red' t "°C"
pause 0.2
reread
To start the live plot (unzip the attachment and CD to it), from terminal first start the bash script, send it to background, then start gnuplot:

Code: [Select]

./tempCPUtoRAM.sh 300 0.2 & gnuplot -e 'set yrange [0:150]; set xrange [0:300]; set title "CPU temperature' settings.gp
To stop the plot and the temperatures harvesting, from terminal CTRL+C to stop gnuplot, then bring the bash script to foreground and CTRL+C it, too:

Code: [Select]

CTRL+C
fg
CTRL+C

Kind of works, though when the refresh speed is too high, sometimes the chart closes, some other times just complains and keeps running.

- Why the sporadic "Skipping data file with no valid points"?
- Other ideas for how to plot a live datastream with gnuplot?

[DiTBho]

what about updates? see here

Code: [Select]

plot "plot.dat" ... blablabla
pause ...
reread


[RoGeorge]

Solved it, there were 2 issues:

- 'reread' in a gnuplot script will mess the line numbers reporting (keeps increasing) in errors and warning messages, to fix this, used replot in a while loop instead

- the tail that was generating the last N samples file 'cpu_temp.txt' was asynchronous with the readings of the same file in the gnuplot, therefore sometimes the gnuplot script was reading the 'cpu_temp.txt' while 'tail' was flushing its buffers into 'cpu_temp.txt', to fix this, the last N samples file 'cpu_temp.txt' was generated inside the gnuplot script, thus guaranteeing the readings only happens after a complete data writing in 'cpu_temp.txt'.


'tempCPUtoRAM.sh' script from the try2 version in the OP is not needed any more, the only script now is this 'plotCPUtemp3.gp', works well even for refresh rates faster than 0.1s:

Code: [Select]

#!/usr/bin/gnuplot

# wxt, x11 and qt support a 'noraise' option
#   to test if with these, the gnuplot script can be closed from GUI: nope
#
# set term x11 1 noraise
# set term qt noraise

# TO DO:  stop the plotting by closing the GUI instead of CTRL+C in terminal

# ! dir <--- this can be between other gnuplot commands on the same line
# system "dir" <--- must be alone in a single gnuplot line

# TO DO:  add trap to delete the work files at exit
# TO DO:  send parameters into gnuplot script
#    command line arguments are ignored in a shebang launch of the gnuplot script
#    either export them then launch the gnuplot script from bash
#    or try assigning them inside the gnuplot script and forget about parameters
#
# for now, use hardcoded values instead
# touch $PLACE/cpu_temp_all.txt > $PLACE/cpu_temp_all.txt
# system "cat $PLACE/cpu_temp_all.txt | tail -n $LASTN > $PLACE/cpu_temp3.txt"
system "touch /dev/shm/cpu_temp_all3.txt > /dev/shm/cpu_temp_all3.txt
system "cat /sys/class/thermal/thermal_zone0/temp >> /dev/shm/cpu_temp_all3.txt && cat /dev/shm/cpu_temp_all3.txt | tail -n 600 > /dev/shm/cpu_temp3.txt"

pause 0.05

# comment out any of the next 2 lines for autoranging
set xrange [0:600]
set yrange [0:150]

set xlabel "Last N samples, each at 50ms apart"
set ylabel "Temperature (°C)"

set title "CPU temperature"
plot "/dev/shm/cpu_temp3.txt" u ($1/1000.0) w l lc 'red' t "CPU (°C)"

# newer 'gnuplot' has 'while' and 'replot'
#   to test if the changes in the datafile can be made live
#   without including the 'plot ' in the loop
#   so it won't mess the gnuplot line numbers in the err messages

# if still gives errors, try generating the data file inside gnuplot
#   using OS commands, in the hope the data update will become
#   synchronous with the pauses and the plotting

while (1) {
    system "cat /sys/class/thermal/thermal_zone0/temp >> /dev/shm/cpu_temp_all3.txt && cat /dev/shm/cpu_temp_all3.txt | tail -n 600 > /dev/shm/cpu_temp3.txt"

    pause 0.05
    replot
}


- to start the live plotting, type ./plotCPUtemp3.gp in a terminal.
- to stop it, type CTRL+C in the same terminal where it was started from, then close the plot GUI, closing only the GUI without CTRL+C first won't make it stop.

Tested on Kubuntu 22.04 LTS, gnuplot v5.4 and a sampling rate/refresh of 50ms, no errors or warnings. 

[julian1]

Sounds like you solved the problem. 

feedgnuplot may also be interesting. I played with it just enough to verify the data windowing function, which is useful if one just wants to plot recent data, rather than everything accumulated from the start of logging.  It's available in some distro repos.

  https://github.com/dkogan/feedgnuplot

  To plot real-time data, pass in the --stream [refreshperiod] option. Data
  will then be plotted as it is received. The plot will be updated every
  refreshperiod seconds. If the period isn't specified, a 1Hz refresh rate is
  used.

And the important bit, is to be able to drop old data, out of the window of interest,

  To plot only the most recent data (instead of all the data), --xlen windowsize             
  can be given. This will create an constantly-updating, scrolling view of the
  recent past. windowsize s

  video
  https://youtu.be/Qvb_uNkFGNQ?t=12830

Edit. Looks like eevblog truncates the youtube timestamp.
At t=12830s
Edit2. defeat forum plugin

[DiTBho]

here it works perfctly as descriped in the article.

[RoGeorge]

Edit. Looks like eevblog truncates the youtube timestamp.
At t=12830s

It's a forum plugin that embeds the video.  The plugin only changes the youtube com URLs, to preserve the timestamp, right click on the video and "Copy video URL art current time" then paste that in the forum.  Youtube will use youtu be domain for this, so the forum plugin won't intervene, though that will show as a link, without the YT thumbnail, yet the video will open at the chosen time when clicked:
https://youtu.be/Qvb_uNkFGNQ?t=12830