Author Topic: Running Seek thermal cameras (& others?) on a Raspberry Pi  (Read 3819 times)

0 Members and 2 Guests are viewing this topic.

Offline IwuzBornanerd

  • Regular Contributor
  • *
  • Posts: 203
  • Country: us
Running Seek thermal cameras (& others?) on a Raspberry Pi
« on: November 05, 2018, 10:02:29 am »
 I have been meaning to start this thread once I get my new SDL2 code ready for release but I decided to start it now because I recently learned that a graduate student (@kiranjayaraj) has been using my old Python code to control a Seek in his research.  He has asked several questions about the code via PM & it occurred to me that the discussion could benefit others who use the program, so this thread can provide a forum for such discussion.

I didn't intend to "support" the program and had even abandoned the Python code because it was incapable of running surveillance while displaying the images at even 1x scale, but knowing that someone has been using that program for such a serious purpose has motivated me to make some relatively easy  updates to the code that people who have used it would probably like to have, so I will post the new code here.  It is also possible that some people might not want to use my new SDL2 code on a Pi because it has to be run in console mode (no X-windows).

I made the topic general enough that we can post links to other software that runs on a Pi.  For instance I know from another thread that people are using this code on GIthub: https://github.com/maartenvds/libseek-thermal which is more of a "bare-bones" program.

The main updates to the Python code have to do with the color palette.  The program now reads in palettes from ASCII file lists of RGB values.  The file names are listed in the config file, so people can create & use their own palettes.  I have included 6 palette files with the program.  Hopefully @frenky doesn't mind that I stole a couple palettes from his software package. :D  There is also a setting for autoranging the temperature use of the palettes wherein the program finds the min & max temperature in each frame and applies the entire palette to that range.  This is, I guess, the common way of doing things and I kind of like it for the smooth palettes but I still like my distinct color palettes without the autoranging.  I have also added the pixel bias correction which makes for cleaner images.  The effect of this was not noticeable with the original Seeks but now that they are much less noisy this correction makes a significant difference when the camera temperature is even just a few degrees different from the temperature at which the "user cal" was done.  To accomplish this the program now saves the shutter data to a file when the "user cal" is done.  There are other things I could or should do but I'll toss this out there now & see how much interest there is otherwise I'll just keep coding and never get it released.  :)  At least with this thread there is a place to comment or complain about the program.

When the palette is in autorange mode the program will display markers at the min & max scene temperatures.  These will update every frame and can appear erratic in some instances (see attached sample video), so I also provided the ability to turn off the min/max marker display.

I have changed the program file names so that compiling the new version (of pixelmath) won't mess up any earlier versions installed on the system.  I also woke up to the fact that there are things I should have changed in the setup.py file instead of leaving the boilerplate stuff there, and I changed some things in that file as well as the file name.  Still, I suggest unzipping the archive to a new directory of your choice.  Once that is done, follow the instructions in the Readme.html file.

I have successfully run my updated version of the Python code on my Pi2B running Raspbian Stretch and on my Pi Zero(W) with a 3.5" touchscreen running Raspbian Jessie (January 2017 version).  I have not gotten Stretch to work with that 3.5" screen but have not spent much time on that.

Attachments below are:
The revised program
Video showing "erratic" markers during palette autorange
Screenshot with "Iron" palette autoranged
Screenshot with my hand-made palette at 5 shades per degree (I like the extra isotherms I get with that palette. ;D )
« Last Edit: November 05, 2018, 10:06:59 am by IwuzBornanerd »
I am not opposed to exercise, unless it is an exercise in futility.
 

Offline Berni

  • Super Contributor
  • ***
  • Posts: 2526
  • Country: si
Re: Running Seek thermal cameras (& others?) on a Raspberry Pi
« Reply #1 on: November 05, 2018, 11:00:33 am »
I just made a similar thing for the Thermal Expert on the RaspberryPi using C++, but its not quite fully featured yet. It just displays the image and that's it.
 

Offline tonykids

  • Regular Contributor
  • *
  • Posts: 65
Re: Running Seek thermal cameras (& others?) on a Raspberry Pi
« Reply #2 on: November 06, 2018, 03:23:34 am »
Just did a quick review of your code ,it seems that you did  a lot of research on different frame IDs :-+
I get an average of 100 frames to make a cal frame and it does better than use one frame,both on compact XR and pro.
 

Offline IwuzBornanerd

  • Regular Contributor
  • *
  • Posts: 203
  • Country: us
Re: Running Seek thermal cameras (& others?) on a Raspberry Pi
« Reply #3 on: November 06, 2018, 07:15:51 am »
I just made a similar thing for the Thermal Expert on the RaspberryPi using C++, but its not quite fully featured yet. It just displays the image and that's it.

Thank you for posting that.  Did you use their SDK?  That was the subject of @kiranjayaraj's initial post (https://www.eevblog.com/forum/thermal-imaging/thermal-expert-raspberry-pi/) and he was having difficulty making progress with it.  I offered to help with the Pi aspects but I don't have a TE camera so I can't be much help with that.

If you are willing to publish your code when ready, feel free to post it on this thread or provide a link to wherever you do post it.  The reason I put "& others?" in my title here was that I thought that people might run into similar problems with the Pi regardless of the brand of camera so this topic could be a sort of gathering place for such information.  Maybe that's not a great idea, but we will see.
I am not opposed to exercise, unless it is an exercise in futility.
 

Offline IwuzBornanerd

  • Regular Contributor
  • *
  • Posts: 203
  • Country: us
Re: Running Seek thermal cameras (& others?) on a Raspberry Pi
« Reply #4 on: November 06, 2018, 07:47:47 am »
Just did a quick review of your code ,it seems that you did  a lot of research on different frame IDs :-+
I get an average of 100 frames to make a cal frame and it does better than use one frame,both on compact XR and pro.

Thanks for the thumbs up.  :)

Looking at the old file directory I see I have averaged up to 100 frames, and 50, and 30 but I don't recall how much difference I noticed, but beware of averaging too many because if you get frames from 2 different shutter periods you will be averaging in some bias change differences and this will mess up your cal factors.  I don't know if the camera ever takes than 60 frames in one shutter period, but I just looked at my 20  minute warmup file & the highest it got to was 60 in that time period.  If the camera is very well warmed up & stable there might only be a few pixels that have changed bias, but I would not bet on any number.  I have observed where pixels toggle back & forth between 2 bias settings every single shutter frame at a certain temperature.  I also wonder if averaging is such a good idea because the image tends to degrade between shutter frames.  Maybe having the non-averaged noise in the cal factors is worse than having the degradation though. :-\

With the lower noise of the newer cameras we really need more precision in those cal factors (in order to get fractional degree accuracy) and that is one of the things I did not yet put in the Python code.  The original camera had 3 digit percentage numbers in frame 10 but the newer ones have 4 (5?) digits that are not simple percentage values and I accommodate that in my SDL2 code.  But merely having more digits there is not truly enough unless we have enough digits in the "diffs" between the external uniform temperature surface & the shutter frame values.  And that requires a temperature difference of something like 60-80 degrees Fahrenheit (double the difference between shutter & outside ambient).  I have not found a good way to get that yet.  If I use something out of the freezer it looks like device temperature gradients show up.  :(
I am not opposed to exercise, unless it is an exercise in futility.
 

Offline Berni

  • Super Contributor
  • ***
  • Posts: 2526
  • Country: si
Re: Running Seek thermal cameras (& others?) on a Raspberry Pi
« Reply #5 on: November 06, 2018, 08:16:22 am »
I used the Linux driver that i got from Thermal Expert via email. For some reason they don't put the drivers on the website. Not quite sure what the deal is there, are the driver not supposed to be available to anyone?

One would think that a thermal camera appears like a webcam, but this is NOT the case, its a fully custom device with a custom driver.

Anyway the technical details of it are that the drivers come as an archive file contains a few *.h C headers and a large *.run file for 32 or 64bit. This run file is actually a shell script that does apt-get on a few USB libraries and then moves the large block of binary mess at the end of the run file into a .so file and copies it into a directory (I think in /etc/). This is a precompiled binary shared object (Equivalent of a DLL in windows). The easiest way to use this .so library is to write a C++ program and link it in and include the provided .h files. This expects you to have knowledge of C compilers and how to write makefiles. Additionally you might sometimes need to set the $LD_LIBRARY_PATH to the right place for things to work(I think the driver sometimes has problems finding the usb library).

Once you have all this you should be able to call the functions from the .h file to make the camera do things. The included doxygen documentation helps with what to do and shows some example code. Using the camera consists of a set of commands that finds the camera(I assume there can be multiple cameras connected) and then initializes it. This initialization part takes a few seconds due to for some reason reading the cameras internal flash really slowly. Once that is done you can call a function that gives you one frame of the image in the form of a float array. The values of the floats are the Celsius temperature for each pixel. For video you simply call this function as fast as possible (And it takes long enough to execute that get you the usual non military frame rate). The other two useful functions are the calibrate and dead pixel correction. You put the included lens cap on the camera and call calibrate to do a flat field calibration and then optionally call the dead pixel correction that maps out and hides any bad pixels. The correction will be included the next time you read a new frame. And that is pretty much everything the driver will let you do (Other functions don't do anything useful).

From here you could create a C++ wrapper app to make it work in Pyhron or <inset language of choice>, or develop the rest of your app in C++.

In my case i opted for doing my whole app in C++ and use OpenGL(using raylib) to draw directly to the screen (Doesn't care if there is X11 running or not, it just steals the frame buffer from it). My goal is to give it similar functionality as one of those standalone FLIR gun shaped thermal cameras plus a few features i think would be cool to have.

 

Offline IwuzBornanerd

  • Regular Contributor
  • *
  • Posts: 203
  • Country: us
Re: Running Seek thermal cameras (& others?) on a Raspberry Pi
« Reply #6 on: November 07, 2018, 08:21:00 am »
Thanks for that information; it should be helpful, but I'll let @kiranjayaraj speak for himself.

I'll have to look into that raylib you used.  I don't recall seeing that before.  SDL2 uses OpenGl also but on the PiZero even that took too long to write to the 3.5" LCD so I ended up using the videocore dispmanx functions to write to the frame buffer.  It also helped to put that in a separate thread since it uses the separate video core.  I don't recall if I tried putting the SDL2 rendering functions in a separate thread.  That might have helped if OpenGL via SDL2 uses the video core...
I am not opposed to exercise, unless it is an exercise in futility.
 

Offline Berni

  • Super Contributor
  • ***
  • Posts: 2526
  • Country: si
Re: Running Seek thermal cameras (& others?) on a Raspberry Pi
« Reply #7 on: November 07, 2018, 09:14:35 am »
With hardware accelerated graphics you sometimes have to experiment a bit to find what works fast. The GPU has a certain way of doing things in a way it prefers and all the layers of software abstraction do too, but in general you usually want to send as few commands to the GPU as possible since the whole path has some overhead. So trying to directly manipulate pixels is usually the worst way.

The way i did it was able to run at 60fps (RPi seams to force Vsync so its capped to 60), this is way faster than needed for a thermal camera. My process was to use a for loop to turn the float temperature data into pixels via a color lookup table. The pixels ended up arranged in a bitmap in RAM and then this bitmap is sent over to the GPU to update a texture in GRAM so that the GPU has access to it, then it simply draws the texture to the screen. This could be potentially speed up even further by writing a shader script for the GPU to do the lookup by itself and we only give it a float array of temperature data, but that's significantly more work and the speed is not needed anyway(The CPU has plenty of time to do the lookup anyway).

In my app i still have to move the camera reading process into a separate thread as the Thermal Expert library blocks inside until the frame is done. Yet you have to call it quickly again to get it to start capturing a new frame as soon as possible. This means doing other stuff in between can jitter the cameras frame rate while at the same time leaving the app waiting in a blocked state for most of the time.

User interfaces in general tend to be a pain to make in just raw C++, but with OpenGL you can make some pretty impressive looking things that run very fast if you know what you are doing.
 

Offline richnormand

  • Supporter
  • ****
  • Posts: 358
  • Country: ca
Re: Running Seek thermal cameras (& others?) on a Raspberry Pi
« Reply #8 on: November 07, 2018, 10:11:45 pm »
Nice job IwuzBornanerd.
I'll give it a run tonight with my Seek Thermal.
Cheers.
 

Offline grizewald

  • Regular Contributor
  • *
  • Posts: 142
  • Country: se
Re: Running Seek thermal cameras (& others?) on a Raspberry Pi
« Reply #9 on: December 17, 2018, 11:16:06 pm »
Thanks for posting this IwuzBornanerd!

I just recently bought a Seek Thermal XR and this code makes a great starting point for running the camera on a Pi 3 B+.

I have it working with the latest Stretch on the Pi and it delivers a solid 8 fps on the more powerful hardware. The first thing I'll be looking at is changing the screen so that I can actually use the buttons on the 3.5" touch screen. They're a bit small for fingers at the moment.  I've not used Python before, but then again, I've used so many languages over my career that another one won't hurt. :D

One language which is my forte is C. When I was tidying up the C module so that I could read it a bit better, a bug jumped out at me. In the shuttercal function, you have a test in two places where you examine each value in the diffs array to see if the program should use that value or the previous one. There's a mistake in that code where you use & and | instead of && and ||. Doing a bitwise operation when you mean to do a logical comparison isn't going to work very well and may explain some of the artefacts I saw under certain conditions.

The corrected code reads:
Code: [Select]
        if (((abs(diffs[ix]) < 6) && (noraster > 0)) ||
            ((badpxls[ix] < 10.5) && (abs(diffs[ix]) > 6) && (pixelcorr > 0)))

Further down in the function, in the clause bounded by "if (bwcolor > 0)", similar code appears, but I suspect this code doesn't do what you wanted it to. It examines a variable "diff" and that is always 1000, as per the initialisation at the start of the function. You might like to have a look at that part to see what is really should be testing!

I'm only just getting into exactly how the different frames work and how the corrections and calibrations are done. If I find anything else, I'll let you know. Once I have my modified version working how I want it, I'll post the changed source files here.
 

Offline IwuzBornanerd

  • Regular Contributor
  • *
  • Posts: 203
  • Country: us
Re: Running Seek thermal cameras (& others?) on a Raspberry Pi
« Reply #10 on: December 18, 2018, 01:39:59 am »
Thank you for finding && pointing that out.  :)  But did you miss the instance right after the line "//calculate scaled (corrected) diffs"?  I think the error comes from having the code originally in Python which did not want the 2 &s.  Really, though, if the comparisons evaluate to 1 or 0 (Isn't that what true & false actually are?) is it any different (I'm no C expert)?  It does work quite well.  Toggle the z key and you see the "patent pixels" come & go.

The "if (bwcolor > 0)" case is defunct & I probably should have removed it since I quit using even before my original release.

Keep looking!  I have a vague memory of finding some sort of calculation error while working on my SDL2 code last year & I lost that code in December when I broke a uSD card.  And I have no idea what the error was in order to recover the correction.  The errors you found were still in that SDL2 code too so I did not find them after I pulled the C module into the all-C code.
I am not opposed to exercise, unless it is an exercise in futility.
 

Offline grizewald

  • Regular Contributor
  • *
  • Posts: 142
  • Country: se
Re: Running Seek thermal cameras (& others?) on a Raspberry Pi
« Reply #11 on: December 18, 2018, 10:46:46 am »
I seem to have corrected the single & in that line without even thinking about it! :)

You're right that it probably works either way, but it does make assumptions about the way the compiler handles the intermediate boolean values from the comparisons, which is always a bad idea. False is always zero, but true is "not zero", so what you get by bitwise combinations of logical comparisons may, or may not be what you expect.

Also, in the test: badpxls[ix] < 10.5, you're comparing an unsigned char (0 - 255) to a float, which isn't really useful as there will never be a fraction in the badpxls array. Changing it to < 11 or <= 10 would make more sense. That's the thing about C - you have to be aware of what types are being used and what the effects of any implicit type conversions might be when it comes to loss of precision or sign. Normally, the compiler will try to do something sensible when you mix types, but it's always best to handle types explicitly and avoid puzzling behaviour.

If you're using & to mean a logical and in Python, then there are similar problems in the Python code. In Python, logical and and or are the words themselves and bitwise and and or are single & and | characters.

That being said, I'm certainly getting some good images from the code! I'll keep you posted if I find any errors.
 

Offline IwuzBornanerd

  • Regular Contributor
  • *
  • Posts: 203
  • Country: us
Re: Running Seek thermal cameras (& others?) on a Raspberry Pi
« Reply #12 on: December 19, 2018, 02:11:05 am »
Also, in the test: badpxls[ix] < 10.5, you're comparing an unsigned char (0 - 255) to a float, which isn't really useful as there will never be a fraction in the badpxls array. Changing it to < 11 or <= 10 would make more sense.
I believe my thinking when I did that was as a way to avoid the = sign, but I don't know why I cared since that number is not all that critical.  It's only there to keep from having ridiculous or zero scaling values on pixels that are not actual image pixels (or maybe dead pixels).

If you're using & to mean a logical and in Python, then there are similar problems in the Python code. In Python, logical and and or are the words themselves and bitwise and and or are single & and | characters.
Now that is just plain funny!  You'd think there was a rudimentary tutorial somewhere in which I would have seen that but I don't remember one.  "And" of all things; whodathunk that?  If I try something and it gets the desired result I move on.  While I can write stuff that works I'm not knowledgeable enough (or perhaps even serious enough) about software to write real good "proper" code, which is why I keep typing disclaimers.  I don't recommend  it for non-nerds.  ;)

And I DO appreciate your nerdy responses, @grizewald.  :)
« Last Edit: April 29, 2019, 10:30:51 pm by IwuzBornanerd »
I am not opposed to exercise, unless it is an exercise in futility.
 

Offline tmorar56

  • Newbie
  • Posts: 1
  • Country: us
Re: Running Seek thermal cameras (& others?) on a Raspberry Pi
« Reply #13 on: May 01, 2019, 09:38:46 pm »
I get link error when I try to make your code on my Raspberry Pi 3 B+. I think I installed the libraries, usb, X11, gl, etc... Could you tell me the Linux commands I need to use in order to solve the link errors. See attached output. Thank you very much for your help.
 

Offline IwuzBornanerd

  • Regular Contributor
  • *
  • Posts: 203
  • Country: us
Re: Running Seek thermal cameras (& others?) on a Raspberry Pi
« Reply #14 on: May 01, 2019, 11:23:13 pm »
I get link error when I try to make your code on my Raspberry Pi 3 B+. I think I installed the libraries, usb, X11, gl, etc... Could you tell me the Linux commands I need to use in order to solve the link errors. See attached output. Thank you very much for your help.

To whom are you addressing your question?  The code I posted above is Python with a C module, no "main.cpp" and no function named ReadRawFrame, and nothing to make other than the c module which has no USB or display stuff in it.  And I don't see where anyone else posted code.  :-//

But as for the errors, my best guess is that you need libusb version 1.0 or higher and the "-dev" packages for libusb, openGl & maybe xserver-xorg.  These contain header files & library files needed for compiling.  Try entering
    sudo apt-get install libusb-1.0-0-dev
and if it does not say already the newest version, then you probably don't have it installed at all. 

If the above clears out the USB errors then you should check the package  manager for the -dev versions of the GL & X packages.  I can't guess which ones you would need so you could just install all of them that you find.
I am not opposed to exercise, unless it is an exercise in futility.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf