Yet another cheap thermal imager incoming.. Seek Thermal

[eneuro]

ZnSe CO2 laser focus lens. these are either Biconvex or Plano Convex. Both will work OK. Plano Convex is likely the best performing
...
ZnSe lense are common in certain thermal camera optics as a high transmission efficiency and relatively cheap element.

Thx for those hints 
If they are usable with laser stuff this is additional bonus.

But what about using them with such MLX90614   Digital, plug & play, infrared thermometer in a TO-can to focus IR on this sensor from bigger area, for example sensor closed in spherical enclusure and such lens in front to try get some kind of average temperature from parallel traces by moveing (rotating in 3 axis) this device by some kind of gimbal with stepper motors?

Point it into different directions and get estimate temperature average without using any thermal camera.
This sensor costs less than 10$, so lens at this price level could be fine if it had a chance to work 

I'll try investigate this andl take a look what available sizes of such lenses are-the bigger diameter the better, I guess for this.

I'm looking forward to seeing what people come up with on the firmware, perhaps the real value is there.

My guess is that next @Mike's teardown Seek Thermal video might be... applying logic analyser to a few of these 18 wires close to the sensor and hacking its communication protocol, while it survived milling and complete lens disassembling and still outputs some decent thermal imaginery

[tmbinc]

If you compare the image quality, for example with a Flir E4, keep in mind that the E4 does *extensive* digital image post-processing.

If you want a fair comparison, try "rset .image.flow.digitalFilter.globEnabled false" on the FLIR; the image gets a LOT crappier, and there's a very visible temperature gradient with the shutter closed, even after a NUC event.

[miguelvp]

Got somewhere, still not stable:

I was aiming for this (done with paintshop pro arithmetic functions):


I got this:


It's kind of an improvement from this:

Other than I loose some details on the grids.

But it only works for about one minute then I loose the reference image and I get a blank screen. In order to use it I have to start the program with the camera pointing down so that it can acquire a reference image of the background noise.

Then per frame it subtracts it and computes the max/min again. I'm not too familiar with managed c, actually I can probably count the times I used c# with both hands. So I have to figure out how to add a keyboard event to the Form, I think I just have to call it Form1_KeyDown or Form1_KeyPress

[callipso]

If the problem is the lens (recategorizing this issue from the minor design flaw category into the total design fail one), the solution to this would be to place the shutter in front of the lens.

[efahrenholz]

Doesn't the flir one use a shutter in front of the lens?

[sgstair]

@miguelvp, I'd add a button control to the form in the UI editor and double click on it to create a new event handler. If you really want the keyboard, in the UI editor properties window, the events tab will let you hook a KeyPress event (or in Form1(), this.KeyPress += <event function>; MSDN has all the details.)
I'm ultimately going to seriously refactor a lot of this stuff, I have some things I'd like to add also.

[miguelvp]

@miguelvp, I'd add a button control to the form in the UI editor and double click on it to create a new event handler. If you really want the keyboard, in the UI editor properties window, the events tab will let you hook a KeyPress event (or in Form1(), this.KeyPress += <event function>; MSDN has all the details.)
I'm ultimately going to seriously refactor a lot of this stuff, I have some things I'd like to add also.

I did that and added a label although it probably should be something other than a label. I did use MFC around 20 years ago, so this is not terribly different other than I forgot (or blocked) all of it.
Feel free to refactor this because I'm truly lost in c#, well not lost as in I can do things with it, but I don't have the energy to study it further.


I'm attaching my project as it's now. I had to make MinValue and MaxValue public and rewritetable and I'm not sure how to pass just the stuff that got modified to add that button. It's still unstable,l it will give up after a while but you can always press the calibrate button to do so.


There is still plenty of work to do, this only calibrates to certain mean temperature (whatever surface you use to do the calibration, other temperatures will see the banding, I think it can be characterized for different hardware but I'm not setup for that)
.
The first calibration goes away, but now if you click the button it will calibrate it again and I didn't see that one time out yet.

So please clean it up and refactor it any way you see fit, I'm not proficient with c# at al.

Next thing is a legend with temperatures based on what the original sensor sees. (kind of same as min max, but for temperatures)

[bktemp]

I would not convert the data to 8bit before applying the palette: The hot metal palette allows to display about 1024 different colours. Especially after scaling the image to double size, this gives a much more smooth display and shows more details than rounding to 8bit.
These are old pictures I took several years ago with a quite expensive camera, so there are not really comparable to seek images, but they clearly show the difference between palettes. The first one is the original image from the camera, the others are processed images using the 16bit thermal data. The last image uses almost the same palatte as the original picture, but it shows much more details. It uses 1536 different colours instead of 256 and it is upscaled to double size. It looks a bit strange compared to hot metal, but it shows the most details.
Upscaling the image data by 2 makes a big difference. The finer details are easily visible. I did use a lanczos scaling algorithm, it is really great for upscaling image data.

[frenky]

My opinion for best way to dead pixel removal and noise control.

Dead pixel problem:
Take an image from thermally equal surface, convert it into black and white. Mark position of all dead pixels and replace it like this:
Get values from top, bottom, left and right neighbor pixel. Check which pair (horizontal or vertical) has smallest difference in color.
Calculated pixel should have the average value of this pair. This will work great on small vertical or horizontal changes in temperature.
(My code already implements this)

Noise control:
One way is to subtract image taken from thermally equal surface (I think miguelvp is already doing this.)
The other more complex way is this: http://ofb.net/~ania/professional/vibret/
So physically rotate module so that each pixel makes a circle then calculate average pixel value in each position.
(This could be also used to increase resolution)
We could use sensors in the phone to calculate offset for each captured frame.

Have fun,
Frenky

[bktemp]

My opinion for best way to dead pixel removal and noise control.

Dead pixel problem:
Take an image from thermally equal surface, convert it into black and white. Mark position of all dead pixels and replace it like this:
Get values from top, bottom, left and right neighbor pixel. Check which pair (horizontal or vertical) has smallest difference in color.
Calculated pixel should have the average value of this pair. This will work great on small vertical or horizontal changes in temperature.
(My code already implements this)

If you use multiple frames from a moving image then this should be possible in background without any need for a manual calibration:
- Detect dead pixels as you described
- Filter the dead pixel map over several frames, but update only when the image is changing (to avoid false masking of good pixels in static scenes)
- Use the generated dead pixel mask to clean the image

It should be even possible to compensate hot pixels (pixels that are not completely dead but have a significant difference in gain, so they can not be compensated with the shutter).
Maybe it could be even possible to predict the drift between shutter calibrations and remove the remaining nonuniformity.

[frenky]

If you use multiple frames from a moving image then this should be possible in background without any need for a manual calibration:
- Detect dead pixels as you described
- Filter the dead pixel map over several frames, but update only when the image is changing (to avoid false masking of good pixels in static scenes)
- Use the generated dead pixel mask to clean the image

It should be even possible to compensate hot pixels (pixels that are not completely dead but have a significant difference in gain, so they can not be compensated with the shutter).
Maybe it could be even possible to predict the drift between shutter calibrations and remove the remaining nonuniformity.

Agreed...

I have found this great lecture on super-resolution:
http://videolectures.net/nipsworkshops2011_sroubek_mobile/

First 20min is mostly theory but I think it's worth seeing...

Must see is:
At 19:42 - implementation of super-resolution in TESTO thermal cameras
At 22:37 - implementation of super-resolution in smartphones (by using gyroscope and accelerometer)

[bktemp]

I only skimmed though the video, but from my experiance with superresolution (both as final product embedded in a digital camera and playing with some algorithms) it works in theory but is not really good for practical usage except with a lot of manual tweaking for each scene.
The main idea behind superresolution is, that the resolution of the sensor is the limiting factor. It is similar to random sampling used in DSOs. But in case of the seek camera the image is quite blurred because of the cheap optics and the avaraging of frames inside the camera. Therefore superresolution is not really usable here. At least it will not give any great improvement without generating unwanted artefacts in the image.

With the nonuniformity (both the gradient and the static noise pattern) and the noise further removed from the data, the seek images should look pretty good.
The processed image of the wifi router looks already quite good compared to the original data.

[eneuro]

I would not convert the data to 8bit before applying the palette: The hot metal palette allows to display about 1024 different colours. Especially after scaling the image to double size, this gives a much more smooth display and shows more details than rounding to 8bit.

Yep, I used this Iron palette at size 256 so far, ONLY for the reason that any sensor available raw data provided there I was able to obtain was from those images included in this thread and they are degraded to RGBA 8bit per channel, but in my libseekthermal.so.1.01 I use DOUBLE matrix in image processing operations, so with acceess to oryginal 14bit sensor data it is easy to convert them to double, make image processing enhancements and then take for example 10bit palette so about 1024 palette size not 256.
That is why my C functions definitions are unsigned int (16bit or more depending on architecture) NOT unsigned char (8bit) when accessing any LUT table 

void seek_thermal_lut_iron256_get(unsigned int idx, unsigned char *pr, unsigned char *pg, unsigned char *pb );
unsigned int seek_thermal_lut_iron256_size();

[mimmus78]

In photography a good way to mask dead pixels is to take a shoot of completely black (with cover on lens) image and completely white image (photo of clear sky with focus set as near as possible) and mask pixels that do not average near black or white.

Don't know if similar way can be used to thermal images too (just use very cold and hot as you can surface).

[Fraser]

Mike,

Regarding the temperature gradient issue.

I thought he same as you when I saw the small aperture at the rear of the lens 'tube'. I am also considering the effect that may be caused when the lens 'tube' touches the microbolometer on one side as appeared to be the case in your video.

If the lens tube is warmer than the microbolometer, contact between the two could potentially cause a localised heat transfer into the microbolometer. The area in contact and adjacent would be warmed, but the opposite side of the microbolometer is adjacent to the shutter access hole and may stay cooler as a result due to air cooling. As the camera warms up, the lens tube may continue to rise in temperature and cause greater temperature differential impact on the microbolometer. Just a thought.

Aurora

[mikeselectricstuff]

Mike,

Regarding the temperature gradient issue.

I thought he same as you when I saw teh small aperture at the rear of the lens 'tube'. I am slo considering the eefect that may be caused when teh lens 'tube' touches teh microbolometer on one side as appeared to be the case in your video.

If the lens tube is warmer than the microbolometer, contact between teh two could potentially cause a localised heat transfer into the microbolometer. The area in contact and adjacent would be warmed but the opposite side of the microbolometer is adjacent to the shutter access hole and may stay cooler as a result due to air cooling. As the camera warms up, teh lens tube may continue to rise in temperature and cause greater temperature differential impact on teh microbolometer. Just a thought.

Aurora

If that was the case, the effect would be the same with the shutter open or closed.
 
My guess is that as the lens holder appeared to be pretty much touching the sensor, it could be that there wasn't enough room for it to get into the exact position it needed, and was endstopped and mounted in a "good enough" position.
Before experimenting with position, I ground out the inside wall to allow a wider range of positions, but hard to say how the "good" position I found related to the original one.
 
The ongoing slow shipping may indicate they are having production issues, maybe this,  and are willing to ship whatever they can get to work reasonably well. The lensholder is a casting, so not easy to make quick changes to.

[frenky]

I only skimmed though the video, but from my experience with superresolution (both as final product embedded in a digital camera and playing with some algorithms) it works in theory but is not really good for practical usage except with a lot of manual tweaking for each scene.

The man having the lecture was obviously involved in developing this feature in TESTO thermal cameras:


So they did managed to implement it into a real product.

I've been playing with it too and results are encouraging. (See attachment)
But I agree that its a big difference doing it on simulated downgraded images or in the real world...

[bktemp]

I've been playing with it too and results are encouraging. (See attachement)

Wow, that looks great! The resolution fits, but they look too good to be images from the seek camera.

[frenky]

No they aren't because I don't have the module yet.
What I have done:
- Found hi-res thermal image online
- make 4 crops with small offset
- resize all 4 images to 50%
- use very basic super-res technique to build hi-res file from 4 small

So this was just for fun...
With the real images the biggest challenge will be to get correct offset between frames.

[bktemp]

No they aren't because I don't have the module yet.
What I have done:
- resize all 4 images to 50%

That is exactly what most people do: Taking an image, resizing it and using superresolution to recreate the original file. This works well, but with real images it gets hard.
I have downloaded some superresolution demos. Using the supplied downsized images, the results were surprising. But using own images from a low resolution camera did not work at all, because they were not sharp enough. Therefore I doubt it will work very well with the seek camera, but it would be great.

[miguelvp]

I did play around with wavelets a while back, maybe I should use that to get rid off the noise at the expense of the edges being altered a bit. My implementation was 2 dimensional for the compression and I was using it for edge detection, although it didn't do that great on some situations.

The processing time is not  terrible but I think to do all that I need to go to C++.

Also my crude attempts to store the mask is only good for the temperature you are focusing on, for example if you want to measure flesh without banding you can calibrate the camera on that range and the banding disappears on the flesh, but the banding appears on the background. The opposite is true as well, if you calibrate it using background temp, then the banding disappears but shows up in flesh.

Seems to be linear but the data should be scaled depending on the temperature, so that the banding disappears for all frequencies.

[miguelvp]

So I'm thinking on doing the following:

Let the warm up and take a series of reference shots of 0 degrees Celsius (ice). Then take a series of shots of 100 degrees Celsius (boiling water). Use that data to compute a per pixel linear interpolation based on the data, this should take care of dead and reference pixels as well because the slope will be 0.

There is at least one special case where a reference pixel actually carries data from the camera. Then make tests to see if the response of the sensor is linear or not, if not linear then I guess I'll have to add more data points to characterize the sensor, I can borrow an IR reader to verify if it's linear or not.

I'm not abandoning the wavelets either because since they work in frequency domain it can be used to enhance the image further.

[frenky]

I also have a plans for this thing...
I'll make a small X-Y cnc like platform and mount it on a stand.
Hardware:
- arduino
- 2 stepper motors from DVD drive http://images.elektroda.net/85_1309460292.jpg
- 2 stepper motor drivers http://goo.gl/JvFfpG

So I'll be able to precisely move the module while taking photos.
With combined data (offset distance+raw image data) it should be relatively easy to remove all the noise and even do some super resolution.

[kwallen]

Is there enough resolution in these devices to do stereoscopic thermal video?

[Fraser]

I just visited the SEEK facebook page. There seem to be happy customers there and pictures look pretty reasonable for $199 !

https://www.facebook.com/pages/Seek-Thermal/628554970550411

Interesting that people with order numbers in the 1700's are now getting their cameras.

Aurora