Author Topic: Thermal Expert- Raspberry Pi  (Read 3132 times)

0 Members and 1 Guest are viewing this topic.

Offline kiranjayaraj

  • Newbie
  • Posts: 2
  • Country: nl
Thermal Expert- Raspberry Pi
« on: September 06, 2018, 11:16:51 pm »
Hello guys,
I have been reading this blog for a few months for information related to my university project. We are building a canopy temperature sensor using IR camera and we did an implementation with a Seek camera on Raspberry Pi.
I recently got hold of a new Thermal Expert Q1, which I am now trying to integrate to my Raspberry Pi and I'm not having the best times with it. I see that there are a lot of posts about the Q1, but does anybody have a solid way to have the TE interact with the Raspberry Pi. I installed their SDK and tried their instructions on running it on the Pi, but it doesn't seem to work at the moment. I can get to those details in trailing posts, but I was wondering if someone had a simpler test code to use the TE (in Python perhaps)? I am also new to Raspberry Pi and linux environments, so appologies in advance if the question seems wrong. (Noob Alert)
Thanks in advance for your responses.
Kiran
« Last Edit: September 07, 2018, 01:35:21 am by kiranjayaraj »
 
The following users thanked this post: jglomas86

Offline IwuzBornanerd

  • Regular Contributor
  • *
  • Posts: 166
  • Country: us
Re: Thermal Expert- Raspberry Pi
« Reply #1 on: September 20, 2018, 06:48:53 am »
kiranjayaraj,

I'm trying to reply to your PMs but the system says your inbox is full so I'll put a reply here & hope you see it.

You asked if it is normal to not see a /dev/video device for the thermal camera on a Pi and that you have seen a FLIRone being viewed on a webcam app.

Yes, that is normal for a thermal camera because there are no standard V4L (or VFW in Windows) drivers for thermal cameras.  The reason you can plug in most webcams and view the output in various different camera programs is that there is a standard (Video for Windows & Video 4 Linux) that specifies a set of standard function calls that all compliant programs will use to get information from the cameras and each camera vendor provides a driver that accepts and responds to that standard set of function calls.  Thermal cameras are not cheap and common enough for that to have happened yet.  They might get there in a few years, though, especially with the Chinese cameras coming out.  That shared library file provided by I3 is the sort of thing that is needed and then someone would need to either make thermal camera viewer programs or add the capability to existing webcam software.  Around 12 years ago or more you could buy webcams that you had to use their special viewers for because the Windows standard was not fully adopted.  It has been fewer years for Linux, and there might still be some webcams that don't work under Linux because the vendor does not want to give out proprietary information.

I suspect that what you saw for the FLIRone is what the guys developed on the " Question about FLIR One for Android" topic(?)  What they did is essentially make a V4L driver for the FLRIone--exactly what Ben321 wanted except for Linux instead of Windows.  :D 

But they used a package called "vloopback", which as far as I know is not officially supported anymore.

Seek does not provide any sort of driver, dll, or shared library like I3 does so there is nothing but hacked information to use with Seek.
I am not opposed to exercise, unless it is an exercise in futility.
 
The following users thanked this post: jglomas86

Offline mahony

  • Regular Contributor
  • *
  • Posts: 139
  • Country: de
Re: Thermal Expert- Raspberry Pi
« Reply #2 on: December 02, 2018, 05:38:04 am »
Hi, late reply but I played a bit with the Raspi Zero and the TE-Q1.
Attached is some very basic code to get images from the Thermal Expert Q1 - for the moment this is really only getting 50 frames as they are from the camera and dumping them to disk. It's just a main.cpp plus simple makefile. The code was tested on a desktop machine (and written) and a Pi Zero W - compiles and runs on both without issues BUT on the Pi I had to run it as sudo for whatever reason. Only requirements are make, gcc/g++ compiler and libusb-1.0 (I used sudo apt-get install libusb-1.0-0-dev).

For any further image processing (dead-pixel removal, factory shutterless correction, manual NUC and so on) you either have to write your own stuff or wait until I get on with this and post an update. ;)

This is NOT using the supplied TE-SDK but I probably also build a version using it at some point just to compare.
« Last Edit: December 02, 2018, 05:39:47 am by mahony »
 
The following users thanked this post: jglomas86

Offline Vipitis

  • Frequent Contributor
  • **
  • Posts: 526
  • Country: de
  • aspiring thermal photography enthusiast
Re: Thermal Expert- Raspberry Pi
« Reply #3 on: December 02, 2018, 06:32:03 am »
what kind of framerate are expectable from a very very basic Pi or Pi Zero?
 

Offline mahony

  • Regular Contributor
  • *
  • Posts: 139
  • Country: de
Re: Thermal Expert- Raspberry Pi
« Reply #4 on: December 02, 2018, 11:08:17 pm »
Hi vipitis,
I did not actually measure the time but it just took a few seconds to grab the 50 frames. I assume you can easily achieve the default 8.5 Hz even on a PI Zero (W). I mean the data rate is really low (384x296x2bytesx8.5Hz = ~2MBytes/s) and even the basic image corrections computations should be doable - it is just a simple lookup and multiply-add per pixel (for simple gain-offset) or a 4th order Polynom per pixel (factory shutterless gain-offset correction).

I can probably do some timing tests this week. I want to extend the code anyway to at least replace the dead pixels and maybe also do the 4th-order gain-offset correction plus show the final image in a simple OpenGL window (not sure if OpenGL will do a the PI Zero - I never tried so far).

EDIT: just tried it on the Pi Zero - and I am pretty unimpressed! So far I did not get significantly above 3fps with dead pixel replacing AND full shutterless correction - no matter if the frames where just read or also dumped to disk and also independent from execution in either pure console or from the desktop. I did not yet get the frames displayed in a GL window on the Pi - seems like I have choosen an improper pixel format or something.

I may do some tests on a Pi 3B+ too but that is not the platform(s) I am really interested in Linux-wise so do not expect too much effort here - I have already skrewed the afternoon getting the GL libs installed on the Pi just to find out my code is not running as is ...  :palm:
« Last Edit: December 03, 2018, 06:07:22 am by mahony »
 

Offline mahony

  • Regular Contributor
  • *
  • Posts: 139
  • Country: de
Re: Thermal Expert- Raspberry Pi
« Reply #5 on: December 04, 2018, 05:57:33 am »
Finally got some GL  code running on the Pi Zero W and Pi3B (so 1.2 GHz NOT 1.4) and adapted my code for the Thermal Expert - it is now running smooth on both.

The 3B achieves somewhere around 8.5Hz at roughly 40% CPU load while the Pi Zero W finally did slightly above 4 fps but almost maxed out on CPU. I enabled the full MKS OpenGL driver on both and had to install some GL libraries on the Zero.

The code is attached plus some actual screenshot of the app running on the 3B. What is still missing is some code cleanup (i.e. move stuff to appropriate classes) and reading the dead pixel map, gains and offsets from camera instead of reading them from disk.

« Last Edit: December 04, 2018, 05:59:35 am by mahony »
 
The following users thanked this post: jglomas86

Offline Vipitis

  • Frequent Contributor
  • **
  • Posts: 526
  • Country: de
  • aspiring thermal photography enthusiast
Re: Thermal Expert- Raspberry Pi
« Reply #6 on: December 04, 2018, 06:58:13 am »
Is your imager limited to 8.5hz or do you have the unlocked 30hz version?

I am just trying to figure out if a DIY handheld monocular would be possible with a small but open computer.
 
The following users thanked this post: jeng

Offline IwuzBornanerd

  • Regular Contributor
  • *
  • Posts: 166
  • Country: us
Re: Thermal Expert- Raspberry Pi
« Reply #7 on: December 04, 2018, 09:05:20 am »
...BUT on the Pi I had to run it as sudo for whatever reason.

In order to NOT need root (sudo) privileges you need a file in the /etc/udev/rules.d/ directory that contains a line like
Code: [Select]
SUBSYSTEM=="usb", ATTRS{idVendor}=="I3vendorID", ATTRS{idProduct}=="TEQ1ID", GROUP="plugdev"Where I3vendorID is the vendor ID number for the camera and TEQ!ID is the ID number for the camera.  You can get the numbers from the command lsusb -v

I don't think the file name matters, but this file was somehow auto-generated with the file name 50-seekthermal-usb.rules for my Seek on my laptop running Debian "Wheezy".  It has not been automagiacally done on any of my Pi's.

And yes you do need at least libusb-1.0-0-dev (and the library itself), older versions won't work.

Does the Zero allow you to enable the REAL MKS driver now, or is it the "fake" MKS?  "Raspbian Jessie" did not work with it (at all), I think.

If you are better at multi-threading than I am you might be able to get some improvement if you put the USB transfer code in a separate thread and also the display writing in a separate thread.  I have to have the display stuff (using dispmanx video core commands) in a separate thread on the Zero in order to get 8fps off a plain non-pro Seek.  Since there is only one CPU core I have not tried putting the USB stuff in a separate thread.  I see no chance of getting any where near 30fps on a Pi though.
I am not opposed to exercise, unless it is an exercise in futility.
 

Offline tmbinc

  • Regular Contributor
  • *
  • Posts: 172
Re: Thermal Expert- Raspberry Pi
« Reply #8 on: December 04, 2018, 10:12:18 am »
I have a custom core (I _will_ follow with details at some point) on a Raspberry PI. My setup is an OpenVizsla board (random FPGA board that I had lying around) receiving LVDS data (FPD-Link II; an interesting topic in itself), feeding that over an FT2232H, and displaying that on a RaspPi (3B, non-plus). The FPGA packetizes the LVDS data and reduces the data slightly (FPD-Link II is 24 bit/pixel payload, but 16-bit are sufficient; 14-bit of thermal data + 2 bits for sync). The RaspPi receives the data (via scanlime's "fastftdi", https://github.com/openvizsla/ov_ftdi/blob/master/software/host/fastftdi.c ), and does the pixel processing (all in software).

I then either output that in a eglfs SDL2 image, or via eglfs qt. (qt because I wanted to provide a UI, but I struggled there...).

The data is around 324x256 with 30fps, and I can display that in realtime.

Some notes:

- Raspberry Pi's USB (dwc2) is _a piece of crap_. (If you look into how it actually works - it's a horror story. One of the issues is that you need to service the stupid USB controller for every microframe. You literally need a full CPU core just for that - and in fact, that's what's for example Windows 10 IoT on Raspberry Pi is doing. ).
- In Linux, just use multiple cores. I have one USB receive thread, one pixel processing thread, one display thread. They get nicely scheduled across the quad cores.
- My pixel processing is plain C code. No fancy shader. I receive a uint16_t framebuffer[2][325][256] (i.e. double-buffering, so I can fill the framebuffer with USB data while the pixel processing thread is running on it; otherwise I would have to serialize or sync the operations - but hey, memory is plenty!), I do histogram equalization (see https://stackoverflow.com/questions/34126272/histogram-equalization-without-opencv ), which requires 1 pass over the image to build the histogram, 1 pass over the histogram (16384 entries) to build a 14-bit-to-RGB LUT, and then one more pass over the pixel data. Since for a typical image only few distinct pixel values are used, the 16384 * 4 bytes LUT cache performance isn't that bad.) The RGB values I derive from a pre-calculated palette with, say, 1024 entries. (So it's really 14-bit-to-10-bit-to-RGB but that's fair enough)
- The end result is a 32-bit framebuffer that I either poke directly to /dev/fb, or submit via whatever GUI library I'm in the mood for.

The most critical path is the USB bandwidth. 324x256x30 is roughly 2.5 MPixel/s, which works with 16 bit per pixel (~5MByte/s), but not with 24bpp. I prototyped the setup with a Zynq-based ZynqBerry, which worked well, but was too expensive in the end.

I can see to share my code but it's super crappy, and I don't have time to work on it right now.
 

Offline mahony

  • Regular Contributor
  • *
  • Posts: 139
  • Country: de
Re: Thermal Expert- Raspberry Pi
« Reply #9 on: December 04, 2018, 05:48:48 pm »
@Vipitis: I am not yet aware of a 30Hz Q1 (unfortunately ;)) - so it is actually limited to pretty close to 9 Hz. I think the 3B should be capable of handling close to 30 fps from an M1 (with 240x180 Pixels). And even the Zero should be able to get to 8/9 fps with the M1 as the amout of data to be processed is less than half of the Q1. I tested my Code with the V1 yesterday on my desktop so I could try it on the Pi too - but I expect to get noticably less than 30 fps with the 640x480 resolution.

Just another example for the processing: I have a Teensy 3.6 running at just 180 MHz that happily grabs the data from a 160x120 Lepton3 at 26 fps (thats the data rate you have to handle: 1 good frame + 2 junk frames) displays it to a 160x128 SPI TFT (8.7 fps) and also dumps the raw data to SD card (once per second). 

As @tmbinc and @IwuzBornanerd pointed out multithreading may significantly improve Performance on the Pi 3 - most likely not on the Zero as it's a single-core.
320x240 at 30 Hz is already quite demanding for the Pi but sounds absolutely reasonable. On a medium i5 processor PC the 640x480x30 Hz cause little more than 30% CPU load so I would assume the 1.2 GHz Pi might be able to get at least 10-15 fps here - probably even more with some streamlined / multithreaded Code.

I may try this on the Pi when the code gets to this point but my primary target is a Jetson TX1 ... so plenty of power here. ;-)

@IwuzBornarnerd: thanks for pointing me to the rights issue - I assumed something like this but did not really dig into the issue as my main PC is not having it.
Regarding the GL Driver on the PI zero: I just activated it through raspi-config but did not actually check which Driver is really used on the zero. I used a new Image of Raspbian Stretch - maybe there are some changes but I am not sure.
« Last Edit: December 04, 2018, 10:51:47 pm by mahony »
 

Offline jglomas86

  • Newbie
  • Posts: 2
  • Country: nl
Re: Thermal Expert- Raspberry Pi
« Reply #10 on: February 25, 2019, 07:46:38 am »
Thanks for the code Mahoni, you have gone beyond what thermal expert does.  :D
Regarding the .bin outputs and .rad
What fileformat / post-processing do they need to get to an image?
Thanks,
JL
 

Offline mahony

  • Regular Contributor
  • *
  • Posts: 139
  • Country: de
Re: Thermal Expert- Raspberry Pi
« Reply #11 on: February 27, 2019, 06:50:46 am »
Hi,
thanks, the code is far from 'finished' but I am lacking time somehow at the moment for all the fun stuff with thermal imaging, coding and so on. ;)
The .bins are basically a raw dump of the 16-bit pixel data, the .rad format is very similar but has a small header of 28 bytes that holds mostly image width, height, bytes per pixel, and pixel type (8, 16, 32, 64 bit / int or floating point) plus some magic number for id purposes.

Attached is a a C# app that can display those files (plus a lot of image processing stuff that is somehow useful to me) and save the data as displayed to .bmp/.jpg/.tiff or .png files. NOTE: there are two .zip files due to file size limits. The app should run without the OpenTK dlls but there is a menu point "3D Ansicht" which needs them. (Yes the menus are mostly German - I have to re-write it at some point and that is on the TODOs …)
 

Offline jeng

  • Contributor
  • Posts: 9
  • Country: lu
Re: Thermal Expert- Raspberry Pi
« Reply #12 on: March 07, 2019, 08:07:48 pm »
@Mahony,
About libusbTEQ1GL, exactly what I need,
But is it possible to pipe this to gstreamer?
I have no X11 server running on my pi zero.
I want to pipe this in gstreamer an then transfer it via WiFi broadcast.
https://github.com/rodizio1/EZ-WifiBroadcast/wiki
« Last Edit: March 07, 2019, 08:32:40 pm by jeng »
 

Offline Ultrapurple

  • Frequent Contributor
  • **
  • Posts: 422
  • Country: gb
  • Just zis guy, you know?
    • Therm-App Users on Flickr
Re: Thermal Expert- Raspberry Pi
« Reply #13 on: March 08, 2019, 01:30:23 am »
Slightly off topic, I know, but In case it's of any interest, Wi-Fi broadcast is built in to the Therm-App native driver and at least one other third-party app (ThermApp Plus). It runs on Android, rather than the various Linux platforms such as the Pi, but many Android stick-type computers will work fine. The main requirement is that the stick must support USB OTG (not just be able to support a USB keyboard/mouse etc).

If I recall correctly, ThermApp Plus establishes an independent Wi-Fi hotspot (sorry, I can't remember the exact terminology or details; it may possibly be Wi-Fi Direct). I am not sure but I think the stock app requires an internet connection, but is viewable anywhere in the world.

PM me if you want any more info and I'll see what I can dig up.
« Last Edit: March 08, 2019, 02:06:48 am by Ultrapurple »
Rubber bands bridge the gap between WD40 and duct tape.
 

Offline mahony

  • Regular Contributor
  • *
  • Posts: 139
  • Country: de
Re: Thermal Expert- Raspberry Pi
« Reply #14 on: March 08, 2019, 01:32:32 am »
Hi,
I assume this should be possible but I never used either gstreamer nor ez-wifibroadcast. You will most likely have to create a compressed videostream from the raw 'bitmaps' that are displayed. I assume that is what gstreamer is for, if not ffmpeg should do. But I got no idea how to put together all the rest. Data rate is states as <12MBit/s so uncompressed might do to but you have to check your tools how to get what you need.

On the other hand I also interfaced a Lepton3 from the Pi3 some time ago and build a very basic(!) UDP-based server on the Pi that streamed the raw data to some host PC via local WLAN. This worked ok-ish but would need some work to be more reliable/stable.
 

Offline jeng

  • Contributor
  • Posts: 9
  • Country: lu
Re: Thermal Expert- Raspberry Pi
« Reply #15 on: March 08, 2019, 07:00:48 pm »


Hi,
I assume this should be possible but I never used either gstreamer nor ez-wifibroadcast. You will most likely have to create a compressed videostream from the raw 'bitmaps' that are displayed.

Ok, that's what I need, but have a problem.
Im not a developer.
So this would be the difficult part for me.


Gesendet von meinem SM-G950F mit Tapatalk

 

Offline jeng

  • Contributor
  • Posts: 9
  • Country: lu
Re: Thermal Expert- Raspberry Pi
« Reply #16 on: March 08, 2019, 07:16:26 pm »
Slightly off topic, I know, but In case it's of any interest, Wi-Fi broadcast is built in to the Therm-App native driver and at least one other third-party app (ThermApp Plus). It runs on Android, rather than the various Linux platforms such as the Pi, but many Android stick-type computers will work fine. The main requirement is that the stick must support USB OTG (not just be able to support a USB keyboard/mouse etc).

If I recall correctly, ThermApp Plus establishes an independent Wi-Fi hotspot (sorry, I can't remember the exact terminology or details; it may possibly be Wi-Fi Direct). I am not sure but I think the stock app requires an internet connection, but is viewable anywhere in the world.

PM me if you want any more info and I'll see what I can dig up.
Hi,
I can give it a try,
I really need this cam working.
The thermal app you are talking, is the TE-Q1 compatible?

If so,
Have a nanopi a64 board here, I can flash android v6.0 on it to test.

http://linux-sunxi.org/FriendlyARM_NanoPi_A64

Gesendet von meinem SM-G950F mit Tapatalk
« Last Edit: March 08, 2019, 07:18:10 pm by jeng »
 

Offline Ultrapurple

  • Frequent Contributor
  • **
  • Posts: 422
  • Country: gb
  • Just zis guy, you know?
    • Therm-App Users on Flickr
Re: Thermal Expert- Raspberry Pi
« Reply #17 on: March 08, 2019, 09:37:30 pm »
I really need this cam working.
The thermal app you are talking, is the TE-Q1 compatible?

Unfortunately the Therm-App is a different device from the TE-Q1, made by a different company. As far as I know there is no common driver. mahony and others know a lot more about I do about running dongle-type cameras on Linux systems and are better placed to advise on your specific issues. The only reason I mentioned the Therm-App camera range was in case someone else is reading this thread in the future looking for a similar, off-the-shelf answer.

Of course there's nothing to stop you selling one camera and buying another but that generally works out fairly expensive. Engineering a solution to your current setup is likely to be a more satisfactory way to go - but of course I don't know your budget.

I should point out that I am very biased towards Therm-App because they are my main area of practical experience and I like the way the cameras work for me. I am not professionally associated in any way with Therm-App or the parent company Opgal - they do not give me any payment or other benefits).
« Last Edit: March 08, 2019, 09:40:29 pm by Ultrapurple »
Rubber bands bridge the gap between WD40 and duct tape.
 

Offline Requiem for a Dreamcast

  • Newbie
  • Posts: 3
  • Country: fr
Re: Thermal Expert- Raspberry Pi
« Reply #18 on: June 03, 2019, 07:41:33 am »
Finally got some GL  code running on the Pi Zero W and Pi3B (so 1.2 GHz NOT 1.4) and adapted my code for the Thermal Expert - it is now running smooth on both.

The 3B achieves somewhere around 8.5Hz at roughly 40% CPU load while the Pi Zero W finally did slightly above 4 fps but almost maxed out on CPU. I enabled the full MKS OpenGL driver on both and had to install some GL libraries on the Zero.

The code is attached plus some actual screenshot of the app running on the 3B. What is still missing is some code cleanup (i.e. move stuff to appropriate classes) and reading the dead pixel map, gains and offsets from camera instead of reading them from disk.

Mahoney, great job with this code! I have adapted your code to work with the M1, and it runs well enough on the Pi 3B. However, since I don't know how to get the dead pixel map or the gain and offsets from the camera, the values are not scaled properly (the dead pixels throw off the range). I have managed to make my own maps more or less, but they are a hack job. How did you get your proper maps?
 

Offline mahony

  • Regular Contributor
  • *
  • Posts: 139
  • Country: de
Re: Thermal Expert- Raspberry Pi
« Reply #19 on: June 04, 2019, 05:13:37 am »
Hi Requiem,
to be honest I read the shutterless data (gains, offsets, dead pixels plus some calibartion constants) via my more functional C# app and just copied the data over.
But it is relatively easy to read all of the data from the first 26 frames (probably less frames on the M1?) after the camera is connected. I think I described it somewhere in the Thermal Expert Q1 thread - it is more or less the same for the Q1 and the V1 and I think also for the M1 as far as I could see in the disassembled Java code. Most likely the calibration constants are in a different place - thats the main difference between Q1 and V1 from my experience.

I can describe it to you if required or maybe point you to the existing description in the TE thread.
At some point I am planning on porting all functionality of my C# code to C++ for use on the Raspi, Jetson Nano and Jetson TX1 - mostly for performance reasons. ;-)
 

Offline Requiem for a Dreamcast

  • Newbie
  • Posts: 3
  • Country: fr
Re: Thermal Expert- Raspberry Pi
« Reply #20 on: June 04, 2019, 06:58:49 am »
Thanks Mahony, I've gone through your posts and found the ones where you explain what the first frames do. Info is a bit all over a few threads so it can be hard to find everything in one place, but following your posts pointed me to the right one. I'll see if I can figure it out when I get some more free time. Cheers.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf