RPi/SBC/Topdon TC001/clone Tester request (Thermal Camera Redux v0.9.3) Github

[G28]

...

I've got a question about the images below. What is that gradient affect happening to the right image? I was able to enable the 8-bit AGC Histogram Equalization on the Lepton and that's what it looked like.

Each frame of the camera consists of 2 sub-frames.  1 sub-frame is the image and the other sub-frame is the thermal data in Kelvin.

There is 1 temperature value per pixel ( 256 x 192 temps in total ).

This double-wide image shows both the Image ( left ) + Thermal ( right ) sub-frame as indicated by the OSD text "Img - Therm".

The app allows each sub-frame to be displayed individually or jointly.

The app does have an optional  Histogram Equalization filter that can enabled / disabled as well as other optional filters and 37 colormaps.

Hope that answers your question.

[Amish Technician]

I've done some more testing on the RPi zero with the new DRAW_SINGLE_THREAD version to compare performance while dragging both rulers (status hud disabled) with -DFAST_DRAG=0 and -DFAST_DRAG=1

OFFLINE
-DFAST_DRAG=0
1x Double 23.0 fps
2x 19.5 fps
-DFAST_DRAG=1
1x Double 23.3 fps
2x 19.2 fps

ONLINE
-DFAST_DRAG=0
1x Double 11.5 fps
1x 15.9 fps
-DFAST_DRAG=1
1x Double 11.3 fps
1x 16.0 fps

As you can see with DRAW_SINGLE_THREAD, frame rate averages are virtually identical with -DFAST_DRAG=0 or -DFAST_DRAG=1
I've gone back and forth between each to subjectively compare the experience for stuttering and frame rate consistency. Offline there was no benefit to fast drag. Online I think it now slightly reduces the experience: With fast drag there's stuttering while dragging slowly but freezes for a moment while dragging quickly, whereas without fast drag enabled it doesn't freeze.

My recommendations:

RPi 1, RPi Zero (W), RPi Compute Module 1
- Single thread
- Online: No drag
- Offline/Freeze frame: Normal drag

All other RPi models
- Multi thread
- Normal drag both online and offline

Not the conclusion I'd expected to come to as fast drag had measurably helped with the previous version, but it's good that this work around is no longer necessary thanks to the improvements from single threading for RPi zero.

DEFAULT_FLAGS='-DNO_DRAG=0 -DFAST_DRAG=0 -DDRAW_SINGLE_THREAD=1 -DDEFAULT_FONT=0 -DDEFAULT_COLORMAP=4 -DROTATION=0 -DDISPLAY_WIDTH=1920 -DDISPLAY_HEIGHT=1080 -DUSE_CELSIUS=1 -DHUD_ALPHA=0.4'

Code: [Select]

Offline

Startup timings: s 1.880, var 0.712, suo 0.357, pa 1.379, pkb 17.916, nw 1421.720, run 0.638, total 1444.602

1x Double, Dragging both rulers around with status hud disabled
main:thrm:img (1.00 : 0.17 : 0.82) scale 1 fps 22.2 read 0.04 rot 7.05 : [47.32 | 8.00 | 38.81] + 10.88 58.27 0.07 BOTH OFF : h 0.00 v 28.30
main:thrm:img (1.00 : 0.19 : 0.80) scale 1 fps 22.7 read 0.01 rot 4.40 : [24.67 | 4.77 | 19.69] + 6.05 30.76 0.04 BOTH OFF : h 0.00 v 14.07
main:thrm:img (1.00 : 0.19 : 0.80) scale 1 fps 22.9 read 0.01 rot 4.27 : [24.61 | 4.76 | 19.60] + 6.07 30.74 0.06 BOTH OFF : h 0.00 v 14.07
main:thrm:img (1.00 : 0.20 : 0.79) scale 1 fps 22.9 read 0.01 rot 4.26 : [24.84 | 4.86 | 19.74] + 6.13 31.02 0.05 BOTH OFF : h 0.00 v 14.26
main:thrm:img (1.00 : 0.20 : 0.79) scale 1 fps 23.0 read 0.01 rot 4.16 : [24.60 | 4.87 | 19.53] + 6.03 30.67 0.04 BOTH OFF : h 0.01 v 14.16
main:thrm:img (1.00 : 0.20 : 0.80) scale 1 fps 23.0 read 0.01 rot 4.31 : [24.63 | 4.80 | 19.60] + 6.10 30.77 0.04 BOTH OFF : h 0.00 v 14.09
main:thrm:img (1.00 : 0.19 : 0.80) scale 1 fps 23.0 read 0.01 rot 4.29 : [24.83 | 4.80 | 19.81] + 6.12 30.99 0.04 BOTH OFF : h 0.00 v 14.35

2x Dragging both rulers around with status hud disabled
main:thrm:img (1.00 : 0.03 : 0.96) scale 2 fps 19.2 read 0.01 rot 4.24 : [24.45 | 0.62 | 23.58] + 14.08 38.57 0.04 BOTH OFF : h 0.00 v 13.89
main:thrm:img (1.00 : 0.02 : 0.97) scale 2 fps 19.3 read 0.01 rot 4.18 : [24.41 | 0.58 | 23.60] + 13.95 38.41 0.04 BOTH OFF : h 0.00 v 14.22
main:thrm:img (1.00 : 0.02 : 0.96) scale 2 fps 19.4 read 0.01 rot 4.25 : [24.85 | 0.60 | 23.98] + 13.70 38.59 0.04 BOTH OFF : h 0.00 v 14.27
main:thrm:img (1.00 : 0.02 : 0.97) scale 2 fps 19.5 read 0.01 rot 4.20 : [24.63 | 0.59 | 23.81] + 13.69 38.36 0.04 BOTH OFF : h 0.00 v 14.08
main:thrm:img (1.00 : 0.02 : 0.97) scale 2 fps 19.5 read 0.01 rot 4.40 : [24.55 | 0.58 | 23.76] + 13.84 38.43 0.04 BOTH OFF : h 0.00 v 13.94
main:thrm:img (1.00 : 0.02 : 0.96) scale 2 fps 19.5 read 0.01 rot 4.19 : [24.71 | 0.61 | 23.79] + 13.85 38.61 0.04 BOTH OFF : h 0.00 v 14.35
main:thrm:img (1.00 : 0.03 : 0.96) scale 2 fps 19.5 read 0.01 rot 4.05 : [24.72 | 0.64 | 23.85] + 13.84 38.60 0.04 BOTH OFF : h 0.00 v 14.32
main:thrm:img (1.00 : 0.02 : 0.97) scale 2 fps 19.5 read 0.01 rot 4.34 : [24.86 | 0.56 | 24.06] + 13.93 38.84 0.04 BOTH OFF : h 0.00 v 14.29
main:thrm:img (1.00 : 0.02 : 0.96) scale 2 fps 19.5 read 0.01 rot 4.11 : [25.53 | 0.62 | 24.62] + 13.48 39.04 0.04 BOTH OFF : h 0.00 v 15.23


Online

Startup timings: s 2.021, var 0.571, suo 0.410, pa 371.017, pkb 0.712, nw 1546.901, run 0.744, total 1922.376

1x Double, Dragging both rulers around with status hud disabled
main:thrm:img (1.00 : 0.24 : 0.75) scale 1 fps 11.4 read 3.78 rot 5.71 : [89.79 | 21.90 | 67.53] + 7.65 97.51 0.07 BOTH OFF : h 0.01 v 19.61
main:thrm:img (1.00 : 0.25 : 0.75) scale 1 fps 11.5 read 1.85 rot 4.48 : [73.41 | 18.06 | 55.16] + 6.62 80.08 0.05 BOTH OFF : h 0.00 v 14.73
main:thrm:img (1.00 : 0.24 : 0.75) scale 1 fps 11.5 read 1.88 rot 4.59 : [73.41 | 17.93 | 55.28] + 6.57 80.04 0.05 BOTH OFF : h 0.00 v 14.73
main:thrm:img (1.00 : 0.24 : 0.75) scale 1 fps 11.6 read 1.88 rot 4.58 : [73.44 | 17.89 | 55.36] + 6.68 80.18 0.06 BOTH OFF : h 0.00 v 14.75
main:thrm:img (1.00 : 0.24 : 0.75) scale 1 fps 11.5 read 1.90 rot 5.01 : [72.89 | 17.75 | 54.91] + 6.50 79.45 0.05 BOTH OFF : h 0.00 v 14.45
main:thrm:img (1.00 : 0.25 : 0.75) scale 1 fps 11.5 read 1.93 rot 4.64 : [72.91 | 17.97 | 54.70] + 6.55 79.51 0.05 BOTH OFF : h 0.00 v 14.33
main:thrm:img (1.00 : 0.24 : 0.75) scale 1 fps 11.5 read 1.85 rot 4.88 : [73.26 | 17.86 | 55.18] + 6.55 79.87 0.06 BOTH OFF : h 0.00 v 14.51

2x Dragging both rulers around with status hud disabled
main:thrm:img (1.00 : 0.14 : 0.86) scale 2 fps 5.5 read 2.88 rot 7.42 : [205.74 | 28.23 | 177.11] + 16.12 221.97 0.11 BOTH OFF : h 0.00 v 22.07
main:thrm:img (1.00 : 0.11 : 0.88) scale 2 fps 5.5 read 1.92 rot 4.12 : [158.58 | 17.98 | 140.32] + 12.46 171.09 0.05 BOTH OFF : h 0.00 v 10.00

1x Dragging both rulers around with status hud disabled - stuttering / inconsistent movement but still better while dragging quickly compared to the 'fast drag' option
main:thrm:img (1.00 : 0.26 : 0.73) scale 1 fps 15.9 read 2.47 rot 5.83 : [71.48 | 18.81 | 52.36] + 4.66 76.23 0.08 BOTH OFF : h 0.00 v 15.67
main:thrm:img (1.00 : -0.00 : 1.00) scale 1 fps 15.9 read 1.98 rot 4.61 : [-8537.67 | 14.08 | -8551.96] + 3.55 -8534.06 0.06 BOTH OFF : h 0.00 v 10.18
main:thrm:img (1.00 : 0.27 : 0.73) scale 1 fps 15.9 read 1.95 rot 4.50 : [51.72 | 14.01 | 37.51] + 3.52 55.30 0.05 BOTH OFF : h 0.00 v 9.98
main:thrm:img (1.00 : 0.27 : 0.73) scale 1 fps 15.9 read 1.93 rot 4.50 : [52.04 | 14.00 | 37.84] + 3.52 55.61 0.05 BOTH OFF : h 0.00 v 10.28
DEFAULT_FLAGS='-DNO_DRAG=0 -DFAST_DRAG=1 -DDRAW_SINGLE_THREAD=1 -DDEFAULT_FONT=0 -DDEFAULT_COLORMAP=4 -DROTATION=0 -DDISPLAY_WIDTH=1920 -DDISPLAY_HEIGHT=1080 -DUSE_CELSIUS=1 -DHUD_ALPHA=0.4'

Code: [Select]

Offline
Startup timings: s 15.927, var 0.636, suo 7.181, pa 1.102, pkb 16.664, nw 1488.083, run 0.527, total 1530.120
1x Double, Dragging both rulers around with status hud disabled
main:thrm:img (1.00 : 0.11 : 0.88) scale 1 fps 22.3 read 0.05 rot 10.91 : [114.21 | 12.28 | 100.99] + 45.92 160.23 0.10 BOTH OFF : h 0.00 v 61.77
main:thrm:img (1.00 : 0.21 : 0.78) scale 1 fps 22.7 read 0.01 rot 4.28 : [23.33 | 4.87 | 18.23] + 6.16 29.52 0.03 BOTH OFF : h 0.00 v 12.67
main:thrm:img (1.00 : 0.21 : 0.78) scale 1 fps 23.0 read 0.01 rot 4.38 : [22.82 | 4.72 | 17.87] + 6.13 28.99 0.04 BOTH OFF : h 0.00 v 12.40
main:thrm:img (1.00 : 0.21 : 0.78) scale 1 fps 23.1 read 0.01 rot 4.30 : [22.72 | 4.74 | 17.79] + 6.12 28.88 0.04 BOTH OFF : h 0.00 v 12.30
main:thrm:img (1.00 : 0.21 : 0.78) scale 1 fps 23.2 read 0.01 rot 4.50 : [23.08 | 4.85 | 18.04] + 6.16 29.28 0.04 BOTH OFF : h 0.00 v 12.39
main:thrm:img (1.00 : 0.21 : 0.78) scale 1 fps 23.3 read 0.01 rot 4.61 : [23.00 | 4.92 | 17.91] + 6.08 29.12 0.04 BOTH OFF : h 0.00 v 12.14
main:thrm:img (1.00 : 0.21 : 0.78) scale 1 fps 23.3 read 0.01 rot 4.49 : [22.90 | 4.91 | 17.81] + 6.14 29.07 0.04 BOTH OFF : h 0.00 v 12.18
main:thrm:img (1.00 : 0.21 : 0.78) scale 1 fps 23.3 read 0.01 rot 4.59 : [23.00 | 4.95 | 17.87] + 6.18 29.22 0.04 BOTH OFF : h 0.00 v 12.15

2x Dragging both rulers around with status hud disabled
main:thrm:img (1.00 : 0.02 : 0.97) scale 2 fps 18.4 read 0.00 rot 4.13 : [23.61 | 0.51 | 22.81] + 14.04 37.68 0.04 BOTH OFF : h 0.00 v 13.53
main:thrm:img (1.00 : 0.02 : 0.97) scale 2 fps 18.6 read 0.01 rot 4.12 : [24.42 | 0.55 | 23.60] + 13.90 38.36 0.04 BOTH OFF : h 0.00 v 14.36
main:thrm:img (1.00 : 0.02 : 0.97) scale 2 fps 18.8 read 0.00 rot 4.29 : [23.65 | 0.51 | 22.88] + 14.06 37.75 0.04 BOTH OFF : h 0.00 v 13.26
main:thrm:img (1.00 : 0.02 : 0.97) scale 2 fps 18.9 read 0.00 rot 4.09 : [23.79 | 0.53 | 23.03] + 14.13 37.96 0.04 BOTH OFF : h 0.00 v 13.85
main:thrm:img (1.00 : 0.02 : 0.97) scale 2 fps 19.0 read 0.01 rot 4.12 : [23.58 | 0.51 | 22.83] + 14.06 37.69 0.05 BOTH OFF : h 0.00 v 13.15
main:thrm:img (1.00 : 0.02 : 0.97) scale 2 fps 19.1 read 0.01 rot 4.09 : [23.62 | 0.52 | 22.83] + 13.88 37.55 0.04 BOTH OFF : h 0.00 v 13.22
main:thrm:img (1.00 : 0.02 : 0.97) scale 2 fps 19.2 read 0.01 rot 4.10 : [23.92 | 0.51 | 23.18] + 14.25 38.21 0.04 BOTH OFF : h 0.00 v 13.61
main:thrm:img (1.00 : 0.02 : 0.97) scale 2 fps 19.2 read 0.01 rot 4.06 : [23.71 | 0.51 | 22.96] + 14.06 37.81 0.04 BOTH OFF : h 0.00 v 13.42
main:thrm:img (1.00 : 0.02 : 0.97) scale 2 fps 19.2 read 0.00 rot 4.01 : [23.31 | 0.56 | 22.53] + 14.45 37.80 0.04 BOTH OFF : h 0.00 v 13.17


Online

Startup timings: s 1.900, var 0.684, suo 0.390, pa 264.882, pkb 8.979, nw 1513.572, run 0.649, total 1791.056

1x Double, Dragging both rulers around with status hud disabled - poor experience. recommend 'no drag'
main:thrm:img (1.00 : 0.23 : 0.76) scale 1 fps 11.3 read 3.49 rot 5.57 : [90.55 | 21.11 | 69.03] + 7.74 98.35 0.06 BOTH OFF : h 0.00 v 17.15
main:thrm:img (1.00 : 0.24 : 0.75) scale 1 fps 11.4 read 1.73 rot 4.45 : [74.61 | 18.08 | 56.31] + 6.58 81.24 0.06 BOTH OFF : h 0.00 v 11.96
main:thrm:img (1.00 : 0.23 : 0.76) scale 1 fps 11.3 read 1.75 rot 4.75 : [75.64 | 17.71 | 57.71] + 6.57 82.25 0.05 BOTH OFF : h 0.00 v 13.34

1x Dragging both rulers around with status hud disabled - stuttering while dragging slowely, freezes while dragging quickly
main:thrm:img (1.00 : 0.24 : 0.76) scale 1 fps 15.9 read 2.90 rot 8.41 : [111.15 | 26.17 | 84.57] + 8.02 119.25 0.09 BOTH OFF : h 0.00 v 26.77
main:thrm:img (1.00 : 0.27 : 0.73) scale 1 fps 16.0 read 1.80 rot 4.58 : [51.16 | 13.67 | 37.30] + 3.57 54.77 0.05 BOTH OFF : h 0.00 v 9.70
main:thrm:img (1.00 : 0.27 : 0.73) scale 1 fps 16.0 read 1.85 rot 4.40 : [51.59 | 13.78 | 37.57] + 3.49 55.13 0.05 BOTH OFF : h 0.00 v 10.02
main:thrm:img (1.00 : 0.27 : 0.73) scale 1 fps 16.0 read 1.82 rot 4.52 : [50.89 | 13.79 | 36.90] + 3.50 54.44 0.06 BOTH OFF : h 0.00 v 9.41


[Amish Technician]

Arrow keys aren't doing anything for me, I tried with two different keyboard, both were laptop style without a num-pad however. I will try again with a full size keyboard.

": Keypad Up/Down/Left/Right/Center(5) moves rulers"

With a RPi it's not working on a full size keyboard for me either. It IS working on my PC however using the same full size keyboard with the num-pad arrows while num-lock is off.

Is anyone else able to reproduce this issue? I'd be interested to know which layout your keyboard uses, and what device you're running redux on.

On a related note I discovered the 'insert' key performs the same function as the 'c' key, decreases the scale. I haven't found another key that increases scale other than 'd'


To compile redux for my PC I first tried just changing the 'march' CFlag like so: '-mavx2 -msse4.2 -msse4.1 -malign-double -march=native' which compiled without errors but fails to run with "Illegal instruction (core dumped)". I'm using a Ivy Bridge CPU which should support these flags according to this page: https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html
I then used these flags and it's working perfectly: CFLAGS='-march=native -mtune=native -pipe -fomit-frame-pointer'

[5U4GB]

Feedback on the latest snapshot, on an ODroid at least it definitely needs to run as root because most of /dev isn't user-accessible:

Code: [Select]

> ll /dev/video0
crw-rw---- 1 root video 81, 11 Jan 18 03:35 /dev/video0

> ./redux -d 0 -snapshot
openCamera(4065): Opening cameara /dev/video0
Error opening video stream(/dev/video0)
Failed to open /dev/video0: Permission denied

> sudo ./redux -d 0 -snapshot
openCamera(4065): Opening cameara /dev/video0
openCamera(4094): Opened camera /dev/video0
Backend: V4L2

However it's still trying to interact with the X11 system when -snapshot is specified:

Code: [Select]

MoTTY X11 proxy: No authorisation provided
terminate called after throwing an instance of 'cv::Exception'
  what():  OpenCV(4.5.4) ./modules/highgui/src/window_gtk.cpp:635: error: (-2:Unspecified error) Can't initialize GTK backend in function 'cvInitSystem'


[tankslapper]

Feedback on the latest snapshot, on an ODroid at least it definitely needs to run as root because most of /dev isn't user-accessible:

Add your user to the group "video", and you shouldn't need to run it as root.

Do the command:

sudo usermod -a -G video yourusername

and you should have access.

[G28]

Feedback on the latest snapshot, on an ODroid at least it definitely needs to run as root because most of /dev isn't user-accessible:

...

However it's still trying to interact with the X11 system when -snapshot is specified:

Code: [Select]

MoTTY X11 proxy: No authorisation provided
terminate called after throwing an instance of 'cv::Exception'
  what():  OpenCV(4.5.4) ./modules/highgui/src/window_gtk.cpp:635: error: (-2:Unspecified error) Can't initialize GTK backend in function 'cvInitSystem'


The back end is required for the video capture to open the camera.  The code is using CAP_V4L. 

X11 needs to be running to do X11 forwarding.

I could move the [-snapshot [prefix]] before the imshow() and exit, but that is well after the camera is initialized.

Here is the OpenCV architecture.  To access the thermal camera, you need to go through a backend.  The selected backend must be available on your platform AND must be able to pass the raw data.

If you choose CAP_OPENCV_MJPEG, it returns Motion JPEG which is a lossy format.  Lossy formats scath the thermal frame data, rendering it useless for this purpose.   CAP_FFMPEG would have pass the raw data unscathed.   CAP_V4L supports the raw format.

[5U4GB]

The back end is required for the video capture to open the camera.  The code is using CAP_V4L. 

Unfortunately the call it fails in returns essentially useless diagnostics, ./modules/highgui/src/window_gtk.cpp calls gtk_gl_init() which just exits with no useful information ("This function will terminate your program if it was unable to initialize the library for some reason"), a Google search just reveals lots of other people with the same problem and no answers.  I can start an xterm so who knows what it could be...

Does it need a particular OpenCV release?  The current version is 4.9.0 but the one that apt-get installs is 4.5.4, I initially tried building 4.9.0 from source but it seems to require a specific build environment that doesn't match what's on the ODroid device.

[G28]

The back end is required for the video capture to open the camera.  The code is using CAP_V4L. 

...

Does it need a particular OpenCV release?  The current version is 4.9.0 but the one that apt-get installs is 4.5.4, I initially tried building 4.9.0 from source but it seems to require a specific build environment that doesn't match what's on the ODroid device.

I am using 4.5.1, so you are using a newer version than what I have tried.

What does the following return ?

v4l2-ctl --list-devices --all

[G28]

The back end is required for the video capture to open the camera.  The code is using CAP_V4L. 

Unfortunately the call it fails in returns essentially useless diagnostics, ./modules/highgui/src/window_gtk.cpp calls gtk_gl_init() which just exits with no useful information ("This function will terminate your program if it was unable to initialize the library for some reason"), a Google search just reveals lots of other people with the same problem and no answers.  I can start an xterm so who knows what it could be...

Does it need a particular OpenCV release?  The current version is 4.9.0 but the one that apt-get installs is 4.5.4, I initially tried building 4.9.0 from source but it seems to require a specific build environment that doesn't match what's on the ODroid device.

Found this:

https://github.com/opencv/opencv/issues/18461

Code: [Select]

I managed to solve it by making sure when I ssh into aarch64 that I use ssh -Y user@host.
This gives authorization to allow cv2.imshow() to display the window.
gtk_init_check error solved
hope that helps

Another post:

To help others, I found that my .bashrc needed to contain the following:
export DISPLAY=:0.0
previously, it said
export DISPLAY=localhost:0.0


[G28]

2024-01-18: Version 0.9.1 modifications have been pushed to github for weak and single threaded/single core hardware.

BIG THANKS to Amish Technician for all of his RPi model variant testing and regressions. 

The release also contains some minor optimizations for regular multi-threaded hardware as well as some recent requests made by others. 

Thanks to all who have provided feedback.

https://github.com/92es/Thermal-Camera-Redux

[5U4GB]

What does the following return ?

v4l2-ctl --list-devices --all

See my earlier post, it's fine with that:

Code: [Select]

USB Camera (usb-xhci-hcd.0.auto-1.1):
        /dev/video0
        /dev/media0

Driver Info:
        Driver name      : uvcvideo
        Card type        : USB Camera
        Bus info         : usb-xhci-hcd.0.auto-1.1
        Driver version   : 4.9.255
        Capabilities     : 0x84200001
                Video Capture
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04200001
                Video Capture
                Streaming
                Extended Pix Format

(About two more screens of output snipped)


[G28]

Sorry about that.  I wanted to make sure it saw V4L and the proper color format.

Code: [Select]

Entity Info:
        ID               : 0x00000001 (1)
        Name             : USB Camera
        Function         : V4L2 I/O
        Pad 0x01000004   : Sink
          Link 0x0200000d: from remote pad 0x1000007 of entity 'Extension 4' (Unknown V4L2 Sub-Device): Data, Enabled, Immutable
Priority: 2
Video input : 0 (Camera 1: ok)
Format Video Capture:
        Width/Height      : 256/192
        Pixel Format      : 'YUYV' (YUYV 4:2:2)
        Field             : None
        Bytes per Line    : 512
        Size Image        : 98304
        Colorspace        : sRGB
        Transfer Function : Default (maps to sRGB)
        YCbCr/HSV Encoding: Default (maps to ITU-R 601)
        Quantization      : Default (maps to Limited Range)
        Flags             :


[Amish Technician]

I've just made a bit of a breakthrough with the 'CPU challenged' Pi zero v1.3

I'm able to achieve a consistent and very responsive 24.9fps live image at 1x scale with both rulers by setting the colour map to 'none'.  I only get ~15 with the default colour map.

[G28]

I've just made a bit of a breakthrough with the 'CPU challenged' Pi zero v1.3

I'm able to achieve a consistent and very responsive 24.9fps live image at 1x scale with both rulers by setting the colour map to 'none'.  I only get ~15 with the default colour map.

Applying a colormap is a single API call into the OpenCV library / frame which appears to be expensive on the Pi Zero v1.3. 

Applying an inverse colormap makes 2 API calls into the OpenCV library.  Could possibly pre-generate inverse maps on startup so when they are applied at runtime, they only make 1 call / frame.

[Amish Technician]

Applying an inverse colormap makes 2 API calls into the OpenCV library.  Could possibly pre-generate inverse maps on startup so when they are applied at runtime, they only make 1 call / frame.

This probably explains how I'm seeing different FPS with different colourmaps (RPi zero):
'Jet'       18.6fps with overlays disabled, 15.4fps with default overlays.
'Bone'   23.5fps with overlays disabled, 18.7fps with default overlays.
'Winter' 24.7fps with overlays disabled, 20.0fps with default overlays.
'None'   25.0fps with overlays disabled, 24.9fps with default overlays.

[G28]

Applying an inverse colormap makes 2 API calls into the OpenCV library.  Could possibly pre-generate inverse maps on startup so when they are applied at runtime, they only make 1 call / frame.

This probably explains how I'm seeing different FPS with different colourmaps (RPi zero):
'Jet'       18.6fps with overlays disabled, 15.4fps with default overlays.
'Bone'   23.5fps with overlays disabled, 18.7fps with default overlays.
'Winter' 24.7fps with overlays disabled, 20.0fps with default overlays.
'None'   25.0fps with overlays disabled, 24.9fps with default overlays.

Jet and Bone should take the same time. 

"Inv Jet" should take longer (as well as any of the other "Inv XXX" and "Cold" because they make 2 OpenCV calls per frame vs 1.

Changing the interpolation method will also change the amount of CPU cycles used. 

Cubic is the slowest, but also the best image quality.  Try Nearest or Linear to see if it makes any detectable speed difference.

[Amish Technician]

RPi zero 1x online

Cubic
'Jet'       18.6fps with overlays disabled, 15.4fps with default overlays.
'Jet Inv' 18.0fps with overlays disabled, 15.4fps with default overlays.

Linear
'Jet'       19.0fps with overlays disabled, 16.2fps with default overlays.
'Jet Inv' 18.0fps with overlays disabled, 15.2fps with default overlays.

Nearest
'Jet'       18.8fps with overlays disabled, 16.3fps with default overlays.
'Jet Inv' 17.6fps with overlays disabled, 15.2fps with default overlays.

[G28]

Looks like 1 FPS can be had on INV maps by pre-generating them at startup.

[Amish Technician]

For anyone having difficulty getting Thermal Camera Redux working you first need to install the 'libopencv-dev' package. Enter the following in a terminal on Raspberry Pi OS (or Debian derived distro on any other device or PC):

Code: [Select]

sudo apt update && sudo apt install libopencv-dev
Open the 'build_redux' file in the 'src' source directory with a text editor and configure the options for your device, or just replace the contents of this file with my one size fits all 'build_redux' script which should work on PC, all raspberry pi's, and likely most other similar SBCs:

Code: [Select]

#!/bin/bash

# Build Thermal Camera Redux

echo "Building redux and test_driver ..."

rm ./redux

# Change build flags based on your hardware
CFLAGS='-march=native -mtune=native -pipe -fomit-frame-pointer'

# Setup default startup configuration options

# ROTATION options:         0, 90, 180, 270
# USE_CELSIUS options:      0, 1
# DEFAULT_COLORMAP options: 0 - 36
# USE_ASSERT:               0, 1
# HUD_ALPHA:                0.0 = Full Transparent, 1.0 Full Opaque, Default 0.4

#  VGA -  800 x  480 is a max 2X scale
#        1280 x  720 is a max 3X scale
#        1920 x 1080 is a max 5X scale
#        2560 x 1440 is a max 7X scale

# 2560 x 1600 is a max 8X scale
DEFAULT_FLAGS='-DNO_TS=0 -DNO_DRAG=0 -DFAST_DRAG=0 -DDRAW_SINGLE_THREAD=0 -DDEFAULT_FONT=0 -DDEFAULT_COLORMAP=4 -DROTATION=0 -DDISPLAY_WIDTH=1920 -DDISPLAY_HEIGHT=1080 -DUSE_CELSIUS=1 -DHUD_ALPHA=0.4'
#DEFAULT_FLAGS='-DDEFAI:T_FONT=0 -DDEFAULT_COLORMAP=4 -DROTATION=0 -DDISPLAY_WIDTH=1024 -DDISPLAY_HEIGHT=768 -DUSE_CELSIUS=0 -DHUD_ALPHA=0.4'
#DEFAULT_FLAGS='-DDEFAI:T_FONT=0 -DDEFAULT_COLORMAP=4 -DROTATION=0 -DDISPLAY_WIDTH=320 -DDISPLAY_HEIGHT=240 -DUSE_CELSIUS=0 -DHUD_ALPHA=0.4'
#DEFAULT_FLAGS='-DDEFAI:T_FONT=0 -DDEFAULT_COLORMAP=4 -DROTATION=0 -DDISPLAY_WIDTH=1280 -DDISPLAY_HEIGHT=1024 -DUSE_CELSIUS=0 -DHUD_ALPHA=0.4'

# Compile and link Redux Thermal Camera code
g++ -Wall -Wextra -O3 -ffast-math ${DEFAULT_FLAGS} ${CFLAGS} -DUSE_ASSERT=0 -I/usr/include/opencv4 tc001.cpp thread.cpp -o redux -lpthread `pkg-config --cflags --libs opencv4`


# Debug for gprof, valgrind and perf
#g++ -Wall -Wextra -ffast-math -pg -ggdb3 ${DEFAULT_FLAGS} ${CFLAGS} -DUSE_ASSERT=0 -I/usr/include/opencv4 tc001.cpp thread.cpp -o redux -lpthread `pkg-config --cflags --libs opencv4`

gcc -Wall test_driver.c -o test_driver

Change these default flags to =1 for improved performance on Pi Zero 1 (W), Pi 1, computer module 1, or any other SBC with a single threaded CPU:
-DNO_TS=1
-DDRAW_SINGLE_THREAD=1


Then from the directory with the source files run the build script:

Code: [Select]

sudo ./build_redux

If it builds successfully you should now have a new binary file called 'redux' in the same directory. Connect your camera and run redux:

Code: [Select]

./redux -d 0If this fails check file permission and ensure 'redux' is executable. Your camera might not be 'device 0' in which case specify the appropriate number for yours instead:

Code: [Select]

./redux -d 1
A limitation to keep in mind with my script is that the resulting binary may not work if you take it to a different device from the one you build it on, if you build it on a PC and copy it to a raspberry pi, or even between some pi models. The flags in this script haven't resulted in a degradation in performance compared to the flags I had manually specified for each RPi before, that said there are flags I haven't tried yet. I'll update here if I discover any improvements (or if someone more knowledgeable has a suggestion that works better).

Hope this is helpful, feel free to ask if any clarification is required or if you run in to any problems.

[G28]

Amish Tech, did you need any additional commands to install the V4L packages or did the OpenCV install pull those in as well ?

[Amish Technician]

From a fresh install of Pi OS 'libopencv-dev' was the only package I installed.

[G28]

Many have expressed the desire for the various thermal camera apps to be able to lock the camera's auto-ranging so when the camera is panned, the colormap would not jump around making it hard to compare relative color coded temperatures.

Originally, I tried to implement this feature, but was only to get it to half work.  After learning more about the camera and the APIs, I believe I will have this feature available soon with a few twists.

So far, it will have 2 controls.  One control will determine if the locking method uses a static pass band or an adaptive pass band filter.

The static pass band filter will clip temps outside of the original pass band set by the original lock as requested by a TC001 user.

The dynamic pass band grows as the camera encounters outliers, never clips and never shrinks back until reset.

The other control will cycle through various linear/weighted temp to colormap conversion algorithms.  Some colormaps favor one algorithm over another depending on the data one is interested in.  It appears the camera uses a weighted algorithm internally so I will try and provide that emulation as well.

I will probably release a beta version to receive feedback and adjust the code accordingly.

[G28]

Many have expressed the desire for the various thermal camera apps to be able to lock the camera's auto-ranging so when the camera is panned, the colormap would not jump around making it hard to compare relative color coded temperatures.

Originally, I tried to implement this feature, but was only to get it to half work.  After learning more about the camera and the APIs, I believe I will have this feature available soon with a few twists.

So far, it will have 2 controls.  One control will determine if the locking method uses a static pass band or an adaptive pass band filter.

The static pass band filter will clip temps outside of the original pass band set by the original lock as requested by a TC001 user.

The dynamic pass band grows as the camera encounters outliers, never clips and never shrinks back until reset.

The other control will cycle through various linear/weighted temp to colormap conversion algorithms.  Some colormaps favor one algorithm over another depending on the data one is interested in.  It appears the camera uses a weighted algorithm internally so I will try and provide that emulation as well.

I will probably release a beta version to receive feedback and adjust the code accordingly.

Some more previews:

The left window has Auto Ranging Locked while the right window has Auto Ranging Unlocked.  Both windows look the same until panned off the hot spots.

 

The left image retains it's locked colormap in contrast to the camera's auto ranging in the right window making it hard to compare colorized temps around the room.

While the auto-ranging is locked, any of the mapping filters can be changed along with any of the other configurations including changing colormaps, portrait/landscale, scale, fonts, etc.

[Amish Technician]

Thermal Camera Redux running at 2x scale on a RPi Zero 2 W with 800x480 60Hz touchscreen (With TC001 for scale):

2x scale


2x scale full screen


Mock-up of possible overlay layout to make better use of available screen space


This is the mock-up being displayed above. I doubled the width of the gradient scale to 16 pixels, I think it might be slightly too wide now 


I have all the different elements on separate layers in GIMP so I can experiment with it. I'm happy to mock-up any layout suggestions.

[Amish Technician]

Another mock-up, 3x scale with no display size limitation.