SeekThermal - How to correct the image received from the sensor?

[frenky]

A BIG THANKS to you for showing us how to get nice clear images out of the sensor.
That pixel gain stuff with frame ID4 is golden.

[frenky]

Let's continue this topic (https://www.eevblog.com/forum/thermal-imaging/seekofix-new-windows-software-for-seekthermal/msg918372/#msg918372) here:

I have now extracted ID9 values as suggested. The chart really looks the same.

So does this mean that value 2000 is -40*C and value 13316 is 350*C? What about the last "bump" with values from 15819 to 16383?

[IwuzBornanerd]

Let's continue this topic (https://www.eevblog.com/forum/thermal-imaging/seekofix-new-windows-software-for-seekthermal/msg918372/#msg918372) here:

I have now extracted ID9 values as suggested. The chart really looks the same.

So does this mean that value 2000 is -40*C and value 13316 is 350*C? What about the last "bump" with values from 15819 to 16383?

That is the way I have been using it, except I use 626F.   

My data also starts at 2000 & peaks at 13316, with 13316 in pixel 14635.  It works out to something like 21.95 "pixels" per degree Fahrenheit.  The number wasn't any nicer for Celsius so I kept the more familiar (to me) units.

Is your 13316 value in pixel 14635?  It would be nice if everybody has the same thing in frameID9--easier to handle it in software.  I re-load it into an array with the pixel's value as the index and the new array value as 22 times the temperature above -40.  Then once I obtain the "diff" scaled by the frameID10 correction factor, I pull the temperature out of the array element number that matches that "diff".  The "diff" is taken from the location of the sensor/shutter on the curve.

The next key to this is that pixel 1 in the image & cal frames gives an indication of the temperature of, well, "somewhere" in the camera.  I say "somewhere" because it lags the sensor/shutter temperature during warmup and once the camera has stabilized the pixel 1 indicator is around 15(?) degrees F below the sensor/shutter temperature.  The number in pixel 1 drops by about 10 per degree F as the camera warms up and appears to be a difference value from a seemingly arbitrary reference point on the curve.  I currently define that reference point as 7679, empirically determined from tests I have run.  I think that puts the reference point at about 180C.

I have not been able to conjure up any reasons for that squiggle after the discontinuity.  I wonder if it means something, but have no clue what it would be.

[frenky]

Is your 13316 value in pixel 14635? 

Yes it is.
I've attached all ID9 values in text file.

The next key to this is that pixel 1 in the image & cal frames gives an indication of the temperature of, well, "somewhere" in the camera.

Wow that is really interesting and very useful.
I will sure look into it.

[IwuzBornanerd]

Thanks for the upload frenky.  Except for a few extra carriage returns and corrupted numbers at line 6210 and 22785, yours is exactly the same as mine.  I'll put mine up here for good measure.

According to the view counter on your graph image, this page has been popular since you posted that...but nobody else is chiming in to say "Hey mine's different." or otherwise.  Maybe not enough time yet...

[miguelvp]

Thanks for the upload frenky.  Except for a few extra carriage returns and corrupted numbers at line 6210 and 22785, yours is exactly the same as mine.  I'll put mine up here for good measure.

According to the view counter on your graph image, this page has been popular since you posted that...but nobody else is chiming in to say "Hey mine's different." or otherwise.  Maybe not enough time yet...

Not enough time here to look, but I did post a while back the raw data for the first 20 raw frames in this post:
https://www.eevblog.com/forum/thermal-imaging/yet-another-cheap-thermal-imager-incoming/msg545880/#msg545880

ID9 should be the 2nd frame.
They are 16 bits per pixel, little endian. Each frame is 208 by 156, 2 bytes per pixel unsigned int16, so the 2nd frame would start at byte 64896 (if 0 based) and it should be also 64896 bytes long.

Note that the data is from a very old firmware, but that will indicate if it has changed in future releases.

[frenky]

My next step will be to record values for 1 hour of:
- second pixel of calibration frame ID1
- average value of calibration frame ID1
- average value of isothermal plate at room temp

So we can see what is exact correlation between sensor temp and shutter temp.

Then I'll strap Peltier module on the back of the Seek to cool it down and repeat the process while trying too keep it's temp at 10*C (50*F).
Housing is made from magnesium alloy so it should be a good thermal conductor.

[IwuzBornanerd]

You will therefore find the attached file interesting.  I did a similar thing about a week ago, while considering how to track the startup variation.  Rather than taking the average of the entire frame, I chose a pixel for which the correction factor in frameID10 is 100 percent.  That pixel was #118.  For a total of 12000 frames (starting with the very first image frame from a cold camera)  I had the code write the following to a file: frame number, pixel 1 value, pixel #118 value for image & cal frame, difference between the image & cal pixel 118 values, and the value of pixel 40 from the image frame.  Pixel 40 appears to be a frame count from the sensor itself.  It increments 4 for every image frame and 17 when there is a cal frame.  You might toss that in for your run just to watch it roll over at around half an hour.  I didn't get to see that because mine is only about 20 minutes. 

The most surprising thing I see in the file is that the pixel values start out (for 2 frames) in the 6000 range and then jump over 2000 before slowly dropping to below 6800 and then jumping more than 2000 again at frame 1741.  I have not read up on how they implement these sensors, but I am guessing that the way they read them is by applying a known current to the pixel & reading the voltage.  If that is the case then perhaps what they are doing here is when the voltage gets below a certain point they up the current in order to keep the voltage in the upper range of the A/D that is reading it.  After frame 1741 the values never got below 8000, so I didn't see another jump.  I think I saw a jump at the 3rd frame even with a warmed-up camera, but it wasn't as big a jump, and I only ran it 12000 frames one time.

Based on this test, I suspect you will find things stable long before your hour is up.

[frenky]

I know what you mean by pixel values jumping. Here are charts that capture that: https://www.eevblog.com/forum/thermal-imaging/yet-another-cheap-thermal-imager-incoming/msg791796/#msg791796
From those charts (that show 1 hour of data) you can see that Seek needs approximately half an hour to stabilise.
Best seen here:
https://www.eevblog.com/forum/testgear/yet-another-cheap-thermal-imager-incoming/?action=dlattach;attach=179925;image

[IwuzBornanerd]

Oh, I forgot I was going to say you might try using that Peltier module to make a uniform temperature surface & look for the temperature at which the image-cal diffs are around zero--that would be the temperature of the shutter.  You would probably want the diffs scaled by frameID10 factors to be the ones around 0, though.  I used a tray of heated water to sort of zoom in on that value & that is where I got my empirical value for the reference point on the curve.   I think I didn't scale the diffs, though...

I hit "post" and it came back saying someone replied while I was typing, so I'll reply to your reply here...I think I saw those graphs before.  The old "picture is worth 1k words"; it's just that the variation is not fast enough to see in the list of numbers.

[frenky]

Your captured data is interesting... Pixel1 data (sensor temp) goes down by 200 in half an hour while difference between shutter temp and external temp changes by 400 (approx) because of shutter heating up.

[IwuzBornanerd]

Kind of makes you wonder if maybe pixel 1 variation is supposed to be only half of the diff, huh?  But the diff keeps changing after pixel 1 settles down...but they bounce up & down too much.

[frenky]

I'm not at my pc any more so I can't post charts showing diff and pixel1 change. Both looked very similar but pixel1 chart was much smoother.

[frenky]

Top line  shows "increase" in sensor temp.
Bottom line shows difference between shutter and external temp though time.

[IwuzBornanerd]

That makes it look like they both settle at about the same time, except for that drop in diff at the end.  I finally pulled that into LibreOffice here & looked to see if there was a smoothing function to apply to the diff curve but I didn't find one.

I think in his teardown video mikeselectricstuff identified a temperature sensor chip on the back of the board, so I expect that is where the pixel 1 value comes from, and why it is less noisy.  The sensor output seems to have several degrees F of random noise.  Again, the graphs suggest that I should use 2x the change in pixel 1 to adjust the temperature readings.  Doing so would move what I call the sensor (actually shutter) reference point up on the curve--ALMOST to the top of the curve.  If it was exactly at the top or close enough to be reasonable I would say "That's it", but my calculation puts it at 12110 which is about 55*F from the top of the curve.

[frenky]

Today I had time to play with Seek...

I recorded 1.5h of data. Seek was facing black alu plate with temp approx 20*C/68*F.
Log file is in the attachment.

Columns:
- ID1pix1   (Sensor temp, value of second pixel in calibration frame ID1)
- ID1avg   (average value of calibration frame)
- ID3avg   (average value of first image frame after calibration)
- FinalAvg    (average value of final-corrected image with gain data)
- FinalMode (mode of this final frame)


If I take avg of final image and subtract ID1pix1*1.6 from it I get stable temp reading after 5min. So for my Seek this would work. I'm not sure if this is the same for all modules.

Blue is sensor temp (values go down but the temp actually goes up).
Red is difference in avg avalues of calibration and image frame (shutter is heating up).
Green is final image corrected by sensor temp.

Without this pixel1 fix we would have to wait at least 20min for temp readings to stabilise. But with this fix 5min is enough.
(Chart shows 1h of data)

[frenky]

Another interesting observation from this data...

If we look at the ID3avg data we can see as the sensor was heating up the raw values went down even though the actual temperature of the alu plate was constant.
So by actively cooling sensor we should gain on thermal sensitivity. (Probably old news to thermal experts )

[IwuzBornanerd]

You had a long day! 

I was confused for a while but I think I understand now...

The fact that you only captured the cal frames & the one image immediately after them makes it look like the pixel1 value changes faster than mine does.  When I looked at my 20 minute warmup run I thought that pixel1 did not change enough to be used to compensate for the warmup drift because it hardly changes between image frames and the warmup change in the image is mostly due to the infrequency of the cal frames.  The first 4 cal frames last no more than 5 image frames but it then switches to 20 images per cal frame--on my camera at least--and then 60.  You can see by the slope of your chart that the image "diff" changes faster in the first 5 or 10 minutes than the pixel1 value does.  I was looking into tracking the startup change when I ran that 20 minute test.  What I decided to do was use the "corrected" diff from one pixel in consecutive cal frames to decide whether and how much to adjust the temperature.  I don't get as much variation in the image with that scheme, but the temperature is still off until the camera is stable.  The temperature is off anyway now because I keep messing with things.

I see a "red flag" here too, in that your pixel1 is about 1000 less than mine for the same temperature, but the delta from "cold" is the same once the temperature is stabilized.  I was thinking this would be a knowable measure of temperature for all cameras, but I guess we have not found that link yet...

[IwuzBornanerd]

On the topic of improving (or "correcting") the image, I have tried averaging pixels from 4 consecutive frames and did not see any improvement--it only slowed down the dancing pixels, but I get a much nicer image if I average adjacent pixels of the same frame.  I am amazed at how this brings out details rather than obscuring them.

I tried it with averaging in the 4 pixels above & below and to each side (for a total of 5), and then the 4 diagonal pixels, and then all 8 surrounding pixels (for a total of 9).  The 9 pixel average looks less noisy but seems to have a bit less detail--more "mushy".

I originally did the averaging AFTER the curve 9 temperature lookup, but it looks about the same if I do the averaging on the scaled "diffs".

I have uploaded sample images of my test wall corner with my cathedral ceiling; the first is with no averaging, then the 5 pixel "+" configuration, then the 5 pixel "x" configuration, then the 9 pixel average.  The hot spot in the images is the heat register in the ceiling.  I had it "toasty" warm in the house at the time. 

[Bill W]

If averaging sucessive frames does not work, this is potentially very interesting.  Had you made s to average 'live' frames, not duplicates produced to drop to 9Hz ?

If so then this suggests that the main problem is in the 'ffc' frame and is therefor correctable if you can get access to the right point in data processing

Bill

www.fire-tics.co.uk

[IwuzBornanerd]

What I did was what I'd call a moving average of the 9fps camera frames.  I defined a set of arrays, frame1, frame2, frame3, frame4 and as each frameID3 came along I'd shift the saved frames down the line &  put the new one in frame1.  I'd make an average of each pixel across the 4 arrays & use those averaged pixels for the output image.

After making that last post I saw the new activity on https://www.eevblog.com/forum/thermal-imaging/seek-thermal-hardware-modification-to-improve-image-quality/25/ in which frenky got improvement with 20 "stacked" frames.  If by "stacked" he means averaged, then perhaps I did something wrong in my code OR maybe my 1 degree Fahrenheit granularity masks the improvement (OR maybe a mere 4 frames wasn't enough!).  All of my images are one color per degree Fahrenheit (or half degree C if I show temp in C).  I don't know what degree resolution frenky is showing there.  If he's displaying 1-bit "diff" resolution then he has about 3 bits finer than mine.  By the way, frenky, that's quite a nifty cooling system you have there. 

You might have noticed in the images in my preceding post that there is a splotchy pattern in the image, sort of a remaining "fixed pattern noise" even after applying the fremeID10 (or "user generated" scaling factors).  I see the same sort of thing in some of the images people have posted on these Seek threads, such as those posted by havaloc here https://www.eevblog.com/forum/thermal-imaging/seek-vs-flir-one-g2-images/?topicseen .  I have found that this is related to the correction factors, as I get a different pattern if I use my own set as opposed to frameID10.

It has become my obsession to get rid of that pattern but I have yet to be truly successful at it.  I finally just subtracted the actual temperature from the lookup values of each pixel of one frame from a uniform temperature surface and used the remainder as a correction in my software.  Dividing the pixel temperature by the actual temperature to create scaling factors had about the same effect.  This produces a rather nice image, but only at temperatures near the temperature where I made the correction values.  So the pattern is related not only to the correction factors but also to the camera temperature and maybe the scene temperature.

The first image attached below is the wall corner with the 9 pixel average from before AND the pattern subtracted.  The second one is a few feet to the right, the wall above my fireplace, with the pattern.  The third one is that image with the pattern subtracted.

I have pretty much given up on using frameID4, 7, & 10 and am trying to come up with my own correction schemes.  Doing so, I recently discovered that if I take an image of a room temperature surface with the camera stabilized and create a set of pixel correction factors by dividing each "diff" by -400 (or maybe even any number between -400 & -500) I get just as clean an image without doing any pattern subtraction, but still only at temperatures within a few degrees of that particular room ambient.  The non-averaged image is much better with that set of correction factors also.

The 4th image attached is the fireplace wall with those correction factors but NO pattern subtraction.

A bit later I made a similar set of cal factors using 5 averaged frames and that is what I am using now.  The 5th image is a fresh one from just now with the camera hung about 10 inches above my tabletop & set to minimum focus distance.  The 6th image is the same with NO pixel averaging.  Compare those with the averaged & non averaged images in my preceding post! 

As a point of comparison, I edited my code to use the frmeID10 correction factors instead of my latest ones and the last 2 images are the tabletop with those factors and with the 9 pixel averaging (7th image) and without the 9 pixel averaging (8th image).  Since I had to restart the program, I couldn't put the markers back in the same spot.

I find that the pattern returns at other camera temperatures.  The 9th image is the ceiling above the camera with the camera's pixel 1 value at 4192, close to the 4198 value where the cal values were made.  The 10th image is the same ceiling with the camera lying in this laptop's "exhaust", heating it to a pixel 1 value of 4060.  I'm not sure if the temperatures are off due to the camera not being stable yet or what; I didn't note whether it was stable.

I am now trying to figure out what the temperature dependance is; it may be a while...especially since  I need to get something else done.

[frenky]

If by "stacked" he means averaged

Yes I'm using ImageMagick "convert *.png -evaluate-sequence median OUT.png" to get median average of multiple images.

I don't know what degree resolution frenky is showing there.

If I take an image of isothemal plane I get about 5*C of temp difference. That is because of shutter heating up.
It seems that your Seek does not have that issue?

I will put all my efforts now into effectively cooling Seek with heatsink small as possible. (I'll put precise temp sensor inside for realtime temp regulation.)

[IwuzBornanerd]

If by "stacked" he means averaged

Yes I'm using ImageMagick "convert *.png -evaluate-sequence median OUT.png" to get median average of multiple images.

Thank you.  For comparison purposes, do you see noticeable improvement if you only average 4 frames?

I don't know what degree resolution frenky is showing there.

If I take an image of isothemal plane I get about 5*C of temp difference. That is because of shutter heating up.
It seems that your Seek does not have that issue?

I will put all my efforts now into effectively cooling Seek with heatsink small as possible. (I'll put precise temp sensor inside for realtime temp regulation.)

By "resolution" I was referring to how you  assign the colors.  Does a change of 1 in the diff value cause a color change or is it more like a change of 10 in the diff?  OR is it based on a fraction of a degree C?  The image will appear more noisy, of course, if a difference of a tenth of a degree is displayed as a different color.  In my images I only display a different color for differences of 1*F, or half degree C when C is displayed.   Oh, and are you using the frameID9 curve to determine temperature now?

As for whether I have a 5*C variation across the image, I have seen about that much at times, but right now it seems not that bad.  I'm attaching some samples.  You already have the room temp. table top above, here are ice at -14*F (per IR thermometer), water at 40*F from the refrigerator, and water at 145*F from the microwave oven.  The hot water is swirling as it cools so it apparently is not uniform temp anyway.

[frenky]

Thank you.  For comparison purposes, do you see noticeable improvement if you only average 4 frames?

I'm at work so I can't do it right now but I would say that it does show improvement. If its a lot better with 20 frames then it's probably is at least a little better with 4 frames.
But you have to know that I don't average sequenced frames. Instead I take many "first after calibration" frames because this frames have least noise.

By "resolution" I was referring to how you  assign the colors.  Does a change of 1 in the diff value cause a color change or is it more like a change of 10 in the diff?  OR is it based on a fraction of a degree C?  The image will appear more noisy, of course, if a difference of a tenth of a degree is displayed as a different color.  In my images I only display a different color for differences of 1*F, or half degree C when C is displayed. 

I do it like this. If temp range on image is 5 degrees then this 5 degrees get mapped to 1000px color palette which has 1000 different colors. If we say that 1 degree is 40 raw diff then I map 200 raw values to 1000 colors. So each raw value has it's own color.

Oh, and are you using the frameID9 curve to determine temperature now?

No I'm totally focused on cooling sensor.
With cooled sensor I'll get stable values and will be able to to map actual raw values to exact temperatures. No need for sensor to stabilise or to fix values with ID9 frame.

I probably could use ID9 for reference temp for cooling sensor but I prefer more PC independant solution: NTC resistor inside seek -> microcontroller -> mosfet -> peltier

[IwuzBornanerd]

But you have to know that I don't average sequenced frames. Instead I take many "first after calibration" frames because this frames have least noise.

I suspected that.

I do it like this. If temp range on image is 5 degrees then this 5 degrees get mapped to 1000px color palette which has 1000 different colors. If we say that 1 degree is 40 raw diff then I map 200 raw values to 1000 colors. So each raw value has it's own color.

I see.  That's good, and it explains why so much noise shows.  I use one degree per color not to hide the noise but just because it's pointless to show more if the accuracy is not there.  I did finally add a "marker zoom" feature to allow quarter degree C resolution, though, in case I ever get the pattern out of there.

What pixel values are you seeing in frameID 1 & 3 with the camera cold?  14k to 16k range??