Yet another cheap thermal imager incoming.. Seek Thermal

[frenky]

@miguelvp:

Thank you for all the effort you are putting into this.
When I receive my module (next week) I will try to contribute as much as possible too...

[miguelvp]

Not a problem, I like doing this. I didn't get to clean the big dead pixels because  I did get distracted with a different algorithm but I only tried it in that one image.
I'll see tomorrow if it can work on my camera live.

Edit: added an auto ranging picture for comparison with the original on my previous post.

Edit2: @Marshall, any chance you could take a similar image from your other more professional imager?

[frenky]

Wow this last image really looks great!

[kihon]

Hi Guys,

Has anyone tried talking to the USB device directly on an android device using the usb host apis? It would be interesting if it allowed us to write our own app to control the seek.

Edit: Also has anyone considered using a cheap fresnel lens to see if the image focus can be improved?

Thanks again,

Kihon

[eneuro]

Has anyone tried talking to the USB device directly on an android device using the usb host apis?

Not yet, but this dongle works under Android, so it should be piece of cake 

Anyway it looks like FFT gives significant speedups in calibration frame approximation, however need to design better filter-this was only speed test with not optimal FFT image sizes:

seek_thermal: Approximation of calibration frame parameters: #11 (FFT)  +0.000000000 +0.000000000 +0.000000000 +0.000000000 +0.000000000 +0.000000000 +0.000000000 +0.000000000 +0.000000000 +0.000000000 +0.000000000  time: 0.005437720 s

Around 5 ms for calibration frame FFT filtering not so bad 

[frenky]

I applied Median filter (6) to this last image in irfanview.
This could be useful to polish the image after all the processing is done because image is more eye friendly and details are still there:

[slackaz]

Here are some pics around my leaky house...  The pics look great, the fact that I get this much air leaking in is not...

'027 and '327 and '472 are of the sliding door in our breakfast nook
'321 and '339 and '705 are of my kid's bedroom on second story
'334 is ceiling in breakfast nook.

[eneuro]

And while we are talking about code, here is my latest Python code.

Maybe some screenshoots or Seek video recorded using this soft? 
Do you use 256 color thermal LUTs in this app?

Meantime optimized approximation of Seek sensor calibration frames and now it takes only 1ms  on Lenovo ThinkPad Z61t under Linux of course

seek_thermal: Approximation of calibration frame parameters: #65  time: 0.000910133

It quite nice fits into sensor calibration frames:

Temperature gradients in the corners smoothed quickly:

While those approximation methods should be fine it is time put everything together and implement into Seek Thermal raw sensor frames image processing pipes which is independent from Seek USB capturing low level API, so one can even use Python USB code to output raw sensor data from Seek and redirect to this Seek OpenThermal image postprocessing engine with universal thermal user interface...   

[miguelvp]

@miguelvp,

I tried out your new version. For some reason my camera looks like crap. I get extremely noisy image. The external reference frame button doesn't seem to give the expected results. I don't get a smooth image after subtraction. Its got a round circular gradient going on.

I don't know how your images look so good. My sensor must be crap. Maybe lots of bad pixels or something. That would be my luck. Also, I have to rotate my tablet upside down to get proper orientation. Its like the image is flipped sideways?

Could you upload the data.dat and data.txt zipped so I can look at how to clean it up further?
The files should be in the executable's folder and it will contain the first 20 frames of raw data from your last run. Just point it somewhere that has some thermal variations and start the program.  Compressed with zip it should be less than 1000KB which is the limit for uploads.

[eneuro]

Compressed with zip it should be less than 1000KB which is the limit for uploads.

Gmail Add attachments

You can send messages up to 25 megabytes (MB) in size.

So, use email, while 20 frames is very small amount and dificult to test anything.
Probably you could then put on your GIT poject server eg. in tst directory and provide link there if you like, but only a few people are playing with this raw sensor data I guess, so gmail should help you exchange bigger files than 1MB easy

[miguelvp]

It doesn't need to be bigger, actually I only need the first 8 frames with the following IDs
4, 9, 8, 7, 10, 5, 1, 3

the rest, will be just helpful for maybe super resolution work.
3, 3, 6, 1, 3, 3, 3, 3, 3, 6, 1, 3

Btw, any pictures you want to share of your results after calibration? would be interesting to see.

[marshallh]

with NEC F30S

[miguelvp]

Thanks for sharing the images, on that version I'm using frame ID 10 and tweaked it manually to your camera depending on the sensor value but I was just basing it off one image.
I also targeted your three particular clumps by interpolating with weights from neighboring good pixels. So that program won't clean other camera's dead pixels.

I think the trick will be to try frame IDs 10 & 4, because I believe both characterize the slope response on a per pixel level.
Hopefully the relation shows the dead pixels with a slope of 0 or very close to it.

I'll keep on playing with the data I have.

[frenky]

Perhaps you could use this "Calibration frame on flat surface" to mark the locations of the hot/dead pixels?
Once you know exact locations it should be no problem to replace them with neighbour values on the final step...

[miguelvp]

Those calibration frames already have the hot/dead pixels included:

Frame ID 4 with values up to 5000 more or less


Frame ID 10 with values up to 200 more or less


My thinking is that they correspond to either high or low calibration points. No matter what it will be easy to map the inert pixels and interpolate the values based on neighbors (even distant ones) by weighting the pixels that need to be travelled.

There are other frames available at boot time that might be helpful as well. I'm just trying different permutations.

[tom66]

Wow, these images look amazing! How well is motion handled?

[eneuro]

Frame ID 10 with values up to 200 more or less

This frame histogram before hexagon pixels filling looks like this:

Surface is similar to calibration frame after median filter and averaging, but will make better image processing using OpenCV-this is imagej2 processed and ploted frame 0xa (10) image:

It is at the begining of startup sequence after frames 0x4 and 0x8 probably.

There are other frames available at boot time that might be helpful as well.

Those router raw sensor data provided by you earlier, shows frames like this in the sequence they apear:

Code: [Select]

sts2img: Frame 0x4 #1  file: facerouter20frames.raw.000000001.0x4.png  208x156  hexagon bad dots: 6  frame dot10: 4  dot25: 0  dot40: 0
sts2img: Frame 0x8 #2  file: facerouter20frames.raw.000000002.0x8.png  208x156  hexagon bad dots: 18  frame dot10: 8  dot25: 0  dot40: 7
sts2img: Frame 0xa #3  file: facerouter20frames.raw.000000003.0xa.png  208x156  hexagon bad dots: 7  frame dot10: 10  dot25: 0  dot40: 185
sts2img: Frame 0x1 #4  file: facerouter20frames.raw.000000004.0x1.png  208x156  hexagon bad dots: 18  frame dot10: 1  dot25: 0  dot40: 23
sts2img: Frame 0x3 #5  file: facerouter20frames.raw.000000005.0x3.png  208x156  hexagon bad dots: 18  frame dot10: 3  dot25: 0  dot40: 28
sts2img: Frame 0x3 #6  file: facerouter20frames.raw.000000006.0x3.png  208x156  hexagon bad dots: 18  frame dot10: 3  dot25: 0  dot40: 33
sts2img: Frame 0x3 #7  file: facerouter20frames.raw.000000007.0x3.png  208x156  hexagon bad dots: 18  frame dot10: 3  dot25: 0  dot40: 37
sts2img: Frame 0x6 #8  file: facerouter20frames.raw.000000008.0x6.png  208x156  hexagon bad dots: 18  frame dot10: 6  dot25: 0  dot40: 43
sts2img: Frame 0x1 #9  file: facerouter20frames.raw.000000009.0x1.png  208x156  hexagon bad dots: 18  frame dot10: 1  dot25: 0  dot40: 49
sts2img: Frame 0x3 #10  file: facerouter20frames.raw.000000010.0x3.png  208x156  hexagon bad dots: 18  frame dot10: 3  dot25: 0  dot40: 54
sts2img: Frame 0x3 #11  file: facerouter20frames.raw.000000011.0x3.png  208x156  hexagon bad dots: 18  frame dot10: 3  dot25: 0  dot40: 58
sts2img: Frame 0x3 #12  file: facerouter20frames.raw.000000012.0x3.png  208x156  hexagon bad dots: 18  frame dot10: 3  dot25: 0  dot40: 62
sts2img: Frame 0x3 #13  file: facerouter20frames.raw.000000013.0x3.png  208x156  hexagon bad dots: 18  frame dot10: 3  dot25: 0  dot40: 66
sts2img: Frame 0x3 #14  file: facerouter20frames.raw.000000014.0x3.png  208x156  hexagon bad dots: 18  frame dot10: 3  dot25: 0  dot40: 70
sts2img: Frame 0x6 #15  file: facerouter20frames.raw.000000015.0x6.png  208x156  hexagon bad dots: 18  frame dot10: 6  dot25: 0  dot40: 76
Hex bad dots count is amount of pixels which are NOT 0x0000 in the place where perfect hexagon pattern should be (so frame type dot 10 (indexing from 0,1,...10 etc) also included in this stat).
It is interesting that dot40 is 185 for frame type 0xa (10) and has value 0 for frame type 0x4-whatever it is, it increases in frames 0x1,0x3,0x6 in time and has values from 23..82
  is it? Temperature in *F ? 82*F~28*C, but 28*F~ -2*C so I have no idea for the moment-maybe they are doing some kind of moving averaging of this value and at the beginning this data is lower than actual temperature?

Another log from extracting those images from your face down file shows that this dot40 value is similar, but slightly different:

Code: [Select]

sts2img: Frame 0x4 #1  file: facedown20frames.raw.000000001.0x4.png  208x156  hexagon bad dots: 6  frame dot10: 4  dot25: 0  dot40: 0
sts2img: Frame 0x8 #2  file: facedown20frames.raw.000000002.0x8.png  208x156  hexagon bad dots: 18  frame dot10: 8  dot25: 0  dot40: 6
sts2img: Frame 0xa #3  file: facedown20frames.raw.000000003.0xa.png  208x156  hexagon bad dots: 7  frame dot10: 10  dot25: 0  dot40: 185
sts2img: Frame 0x1 #4  file: facedown20frames.raw.000000004.0x1.png  208x156  hexagon bad dots: 18  frame dot10: 1  dot25: 0  dot40: 23
sts2img: Frame 0x3 #5  file: facedown20frames.raw.000000005.0x3.png  208x156  hexagon bad dots: 18  frame dot10: 3  dot25: 0  dot40: 28
sts2img: Frame 0x3 #6  file: facedown20frames.raw.000000006.0x3.png  208x156  hexagon bad dots: 18  frame dot10: 3  dot25: 0  dot40: 33
sts2img: Frame 0x3 #7  file: facedown20frames.raw.000000007.0x3.png  208x156  hexagon bad dots: 18  frame dot10: 3  dot25: 0  dot40: 38
sts2img: Frame 0x6 #8  file: facedown20frames.raw.000000008.0x6.png  208x156  hexagon bad dots: 18  frame dot10: 6  dot25: 0  dot40: 44
sts2img: Frame 0x1 #9  file: facedown20frames.raw.000000009.0x1.png  208x156  hexagon bad dots: 18  frame dot10: 1  dot25: 0  dot40: 50
sts2img: Frame 0x3 #10  file: facedown20frames.raw.000000010.0x3.png  208x156  hexagon bad dots: 18  frame dot10: 3  dot25: 0  dot40: 55
sts2img: Frame 0x3 #11  file: facedown20frames.raw.000000011.0x3.png  208x156  hexagon bad dots: 18  frame dot10: 3  dot25: 0  dot40: 60
sts2img: Frame 0x3 #12  file: facedown20frames.raw.000000012.0x3.png  208x156  hexagon bad dots: 18  frame dot10: 3  dot25: 0  dot40: 64
sts2img: Frame 0x3 #13  file: facedown20frames.raw.000000013.0x3.png  208x156  hexagon bad dots: 18  frame dot10: 3  dot25: 0  dot40: 68
sts2img: Frame 0x3 #14  file: facedown20frames.raw.000000014.0x3.png  208x156  hexagon bad dots: 18  frame dot10: 3  dot25: 0  dot40: 72
sts2img: Frame 0x6 #15  file: facedown20frames.raw.000000015.0x6.png  208x156  hexagon bad dots: 18  frame dot10: 6  dot25: 0  dot40: 78
sts2img: Frame 0x1 #16  file: facedown20frames.raw.000000016.0x1.png  208x156  hexagon bad dots: 18  frame dot10: 1  dot25: 0  dot40: 84
For last frame #16 it has value 84 which means if it were 84*F~29*C, so still it could be temperature inside this Seek dongle close to sensor, but I do not know history how you played with it and what room ambient temperature could be during this USB dump
My suspect is this might be temperature reading from its sensor, but it require making some measurements on real device.

Anyway, it is clear that for testing purposes much longer period of time from startup needed to see how this USB raw data changes in time, so it will nice to have such UCB dump and test on the same raw sensor data diffrent algorithms used in software without loosing time and doing this in realtime.

NOTE: Maybe for the moment I've missed a few frames, due to additional veryfication if it fits into this hexagon black pixels pattern, but 16 frames is close to maximum for this raw data file size: 1297920/(208/156/2)= 20 frames in theory...

Update: That is interesting that 185*F~85*C so very common temperature range we can find in many datasheets.
0*F~ -18*C , so it could be some measurements made during manufacturing etc, but those values for 185 and frame type 0xa are below just 185, so maybe other mapping to expected operating temperatures of this Seek dongle, but do not cracked this for the moment 

[cynfab]

There is more to dot[40] than what you see in the first 20 frames or so. I printed out the value as well as the frame ID and found that the value in dot[40] continues to increase. I'm running that test now and it is up to ~20000 after about 10 minutes of operation. The value is almost the same in both the data frame and the cal frame, so when the frames are subtracted, they cancel out.
The value just passed 32768 so it will probably wrap at 64K.

 I wonder what other interesting things are hidden in the data.

   ...ken...

Edit1... Yep it did wrap at over 8200 frames since start.
Edit2... added dot40.txt which is the beginning and end of the test run.

[eneuro]

I printed out the value as well as the frame ID and found that the value in dot[40] continues to increase. I'm running that test now and it is up to ~20000 after about 10 minutes of operation.

Maybe this is frame time stamp in this dot[40] value?
65534/10min/60s~ 109/s   ~ 100 Hz -> 0.010 s 10ms per 1 increase

Code: [Select]

...
(3, 204.0)
(3, 208.0)
(3, 212.0)
...
(3, 199.0)
(3, 207.0)
(3, 215.0)
...
we have difference about 4 between frames there, so  40ms which could give us 25 frames type 0x3 per second, but there are also other frames type 0x1 .

Yep, at the end of this file we have 8 difference between next frames so we have 73ms per frame ~ 14 fps so close to this Seek thermal FPS limit while we have calibration frames 0x1 between and maybe those 0x6 also, so it will drop down to those 9Hz of output FPS 

My guess is it is probably frame time stamp in dot[40] word value.
Thanks for this text log it helped a lot to figure it out 

Lets examine another hexagon pattern pixels at the begining of each frame 

BTW: Which is real output FPS of this Seek Thermal dongle - 9Hz or less or maybe more?

[efahrenholz]

It stacks 4 frames per real frame, and at 9hz that makes it about 36fps.

[cynfab]

We should probably investigate any of the "patent pixels" that are non-zero.

[eneuro]

We should probably investigate any of the "patent pixels" that are non-zero.

I've already wrote such software and have function which returns all those "patent pixels" - I call them hexagon dots while they looks like this 

./sts2img: Raw sensor input file: sts.raw  frame 0x3 #5  output files: sts.raw.000000005.0x3.png sts.raw.000000005.0x3.img.png  208x156  hexagon bad dots: 18  frame dot10: 3  dot25: 0  dot40: 28

For the moment outputed only those three first: dot[10]  dot[25] whch looked 0x0000 on this short 20 first frames after Seek start, and dot[40].

Bad dots: 18 in log from processing this frame type 0x3 means that there were 18 non zero values in this hexagon patent pixels, so for me it is piece of cake to print where those nonzero pixels are with its coordinates (colX,rowY with [0,0] at first pixel) and index (starting frm 0,1,.. etc) and values after processing each frame, so it could look like this:

Code: [Select]

.... frame #NNNNN ... ( colx: 40  rowy: 0  dot[40]:  0x001C  28 )  ... 

BTW: It would be nice record and provide share link to longer Seek USB raw data sequence, lets say 128MB which could be around 2500 different frames of metal object taken into freezer and put on hot iron to have temperature ranges from <0*C and above human body temperature >40*C 

I haven't got this dongle in my hands for the moment, so if someone could provide such bigger Seek USB raw data we could work on the same data and exchange knowledge of possible protocols used there, while it is quite easy write software in any programming language and its implementation details are not such important, while the most interesting part is to guess what is going there on this USB link when this Seek dongle is powered ON and find out or maybe even unlock capabilites hidden in their oryginal app 

Update:
Just made comparision of frame 0x1 and 0x6 which looks like calibration shutter frames, but which is interesting when substracted 0x6 from 0x1 and added 10000 to avoid saturation in imagej2 image processing tool, but it is interesting that something is going on first row and last row in those Seek sensor frames, while difference between those two frames is very small in other areas but it looks very suspicious in places where hexagon patent pixels are in this first & last rows .


Histograms of those frames attached-this is preprocessed raw sensor data with hexagon pattern and dead pixels detected on calibration frame filled with neighbours average, so no bluring and correction from 207th column.
Those histograms looks the same, but I suggest make closer look to those hexagon patent pixels in this first & last row -it clear that they are changed between 4th 0x1 and 8th 0x6 

Included also histogram of 0x3 image frame from router raw data sequence 

[blackboxdisease]

@eneuro

I won't be using my SEEK for a while, it just performs too awful. It has excessive noise and gradient issues, it could be because here in the cold eastern part of Canada, that I am experiencing more severity of problems, or if my unit is more defective.

I take back what I said earlier about the shutter. I took a very weak fridge magnet and locked the shutter closed, and there was no high/low temperature variation or gradient. Even when subjecting it to very low and high temps. Perhaps it has something to do with the programming and algorithms used.
 
So if you want, I can lend you mine for development purposes.

[eneuro]

@blackboxdisease I've send you PM right now, so we can discuss this via e-mail or Skype, while as I wrote w few people already are interested in their own apps for specyfic thermal imaginery tasks, so I will make some development for them, so can try figure out how your dongle will perform in of course non destructive software tests  after checking in Seek support its recomended operating temperature range, while object temperature which it can detect can be very different

Next week I hope I will setup small Wiki on my http://openthermal.sourceforge.net  web page and forum for people playing with this device at low level USB protocol and we'll try to make working Linux kernel driver to easy acces this device from Linux/Android devices via USB.

We'll see if Seek will make maybe improoved hardware version of this Seek Thermal, while for the moment it looks like it is no longer available via Amazon:
Amazon: Seek UW-AAA Seek Thermal USB Connector, Black Currently unavailable. 

However,  in this Seek Thermal comparision to Flir One USB dongle  http://www.pacbiztimes.com/2014/08/22/competition-heats-up-in-infrared-market-as-flir-seek-thermal-release-smartphone-products  , it looks like a lot of money were invested, and Seek Thermal founders Parrish and Fitzgibbons will heat up competition in thermal imaging dongles of decent quality, since those 2.1k Seek "hexagon pattent pixels" is about half of Flir One 4.8k sensor so 30k Seek real thermal image pixels available via USB looks like should be a huge advantage, and yep Flir One 80x60 array produces in many cases thermal blobs with contures added from high res visual camera, so Seek should easy outperform this Flir One dongle in thermal image resolution.
Need to watch again Flir One teardown to compare its shutter with Seek solution from mechanical and thermodynamics point of view, while Flir E4 shutter looks very solid and perfectly cut off any incoming external IR-simply shuter is a black wall in Flir E4 design 
Anyway it will be interesting if we'll be able to improove Seek output quality just by only playing with software, maybe calibrated and recompiled for specyfic device based on a few additional calibration experiments 

[barehill]

eneuro
 I look forward to your Source forge code. I want to dig into this camera, it is quite a breakthrough! Specifically I don't understand why it will only work on certain devices. Also perhaps improved apk's can be made available by developers. It is only to Seek's advantage. Presumably any proprietary/IP code would reside in firmware.

I would also like to model the optical system to investigate its performance, compare it to other designs, and look for internal stray reflections which could seriously affect performance.