SeekThermal - How to correct the image received from the sensor?

[jadew]

Hey guys,

I'm trying to get some software going for this camera but I'm having a hard time correcting for fixed pattern noise.

The problem is that even if I subtract the calibration frame from subsequent frames, the fixed pattern noise is still there. Flat Field correction was suggested to me, but I'm not entirely sure how to get that working with out manually calibrating the camera.

The mobile app manages to get proper images, which means it is possible. Any ideas on how to achieve the same result?

As soon as the camera is initialized, it sends out 6 frames, which I attached.
Once in a while, it sends 2 calibration frames, one before the shutter is heard, another one after or at the same time - can't tell. Also attached.

Blue pixel = min displayed value
Green pixel = max displayed value

Edit: The last picture is obtained by applying the offset from the calibration frame.

Let me know if you have any ideas. Thanks.

[ranch varment]

Whats that noise even doing there, it looks like it was added by your program.   (it looks computery)

Whats causing it, it actually looks to me like some software fault, not something youd get practically at all, besides i just reread and you said some other software was working -  does it have a similar look to this?   note - I dont know much about thermal cams.

[miguelvp]

Not added noise, it's on the actual sensor:

Read this thread from this point on:
https://www.eevblog.com/forum/testgear/yet-another-cheap-thermal-imager-incoming/msg533735/#msg533735

Be warned, many many pages but with source code etc.

[John Coloccia]

What camera is it?  Exactly what are the cal frames?  Obviously, one is for a dark cal.  Is the other one also dark current? It seems brighter.  Different integration time?  I have a funny feeling the second image might be for gain flattening.  Perhaps some of the initial images are for gain flattening?

Typically, you would take many dark current frames and average them to get your dark cal. Initial Frame 3 looks like a dark cal also.  At a minimum, try averaging the 2 cal frames, and maybe even initial frame 3, and use that as your dark cal.  Worth a shot, anyhow.

Do you have more information, or are you trying to reverse engineer the camera output somehow, and it's some sort of proprietary rubbish?

[miguelvp]

Here are my interpretations of those images but that was back in November:

https://www.eevblog.com/forum/testgear/yet-another-cheap-thermal-imager-incoming/msg545910/#msg545910

Somewhere in there there is a zip file with the code for a windows app that I did based on sgstair's github code:
https://www.eevblog.com/forum/testgear/yet-another-cheap-thermal-imager-incoming/msg534497/#msg534497

mine is buried somewhere in there.

Best image I got with an extra lens:



Someone else's camera using my program but I did make a custom version for him to get rid of his dead pixels:


Here is one of the versions on the zip file on this post
https://www.eevblog.com/forum/testgear/yet-another-cheap-thermal-imager-incoming/msg552223/#msg552223

Not sure if there are more after that one.

[John Coloccia]

Studying it some more, it looks to me like Frame 3 is a dark cal, and frame 4 is flat field...maybe from the factory?  Are those frames identical every time the camera starts up?

If that's true, you can subtract frame 3 from frame 4 to get the pixel gain corrections.

[miguelvp]

John, if you want to play with the images, here are some raw captures zipped on this post, including those first frames and visible frames. One set with the camera facing down, the other one pointing to my Verizon router, where the Z is in red so it doesn't really show in the closeup above, but the raw images are from farther away and without the close up lens I used for the one I embedded above.

https://www.eevblog.com/forum/testgear/yet-another-cheap-thermal-imager-incoming/msg545880/#msg545880

I still have the project somewhere in my system and I'm not entirely sure what frames I used for calibration but I'll try to find the last revision.

[miguelvp]

Don't know if this will help of confuse people more. I used Visiquest (no longer available, that came up from some old software called Khoros) to do my image processing.

This seems to be a test harness that I was using at the time:


It seems I used frame ID 4 (frame 1) and blend it with frame ID 1 subtracted from frame ID 3. Half a year and some work crunch time I don't quite recall what I was doing. But I'm still going to look for the current code.

[miguelvp]

Of course not the same angle but should be close enough, here is what that section of the router (not on the raw data but the one image I linked above) looked like.

The red color of the Verizon logo doesn't seem to dissipate as much heat as the white letters.

[jadew]

Thanks for the replies guys. I'll follow your suggestions and I'll let you know what works and what doesn't.

@Miguel
I think frame ID4 (the first one) is very important too, but the problem is that for me it shows a horizontal line (this frame changes), while the raw frames don't have that sort of distortion.

On the other hand, your program seems to generate very nice images.

@John Coloccia
Frame 3 and 4 (IDs 8 and 7) are very similar between initializations, but they're not identical.

I will give it a try tho and see what happens.

Edit:
I gave that a try, didn't work - maybe I didn't do it properly.

For the record, the way I'm applying the calibration frame (ID 1), is by calculating the offset between each pixel and the average pixel value. I am then adding this offset to the actual frame.

I saved a list of 50 frames, in the order in which they are received, for anyone who feels like taking a look. They're saved as 206 x 156 x 16 bit, so every 206 x 2 bytes represents a row. Originally, there are 208 pixels per row, but the last two pixels are useless so I trimmed that out.

As sgstair discovered, pixel 10 (the first dead pixel in the repeating non-noise pattern), is the status/ID byte.

In order to identify a non-noise dead pixel from the repeating pattern you can use the following function:

Code: [Select]

static bool is_pattern_pixel(int x, int y)
{
int pattern_start = (10 - y * 4) % 15;

if (pattern_start < 0)
pattern_start = 15 + pattern_start;

return x >= pattern_start && ((x - pattern_start) % 15) == 0;
}


[miguelvp]

@jadew

I think your result looks very clean, you just need to do a bilinear interpolation or some kind of interpolation to despeckle the image.

[jadew]

@miguelvp

Yeah, it doesn't look bad, but I think it's possible to increase the image quality based on the calibration data. My guess is that once we do that properly we'll end up with a much better image that we can further process.

The rest of the 50 frames set is attached.

[jadew]

Ok, I figured it out, but it turns out miguel already said that this is the case. Frame 1 (ID 4) is the gain compensation frame.

[miguelvp]

That looks nice, any chance you share the code?

[jadew]

I'm going to write a program in Qt and share it when that's done, so in a couple of days. Currently I'm making use of proprietary libraries that I can't opensource.

[John Coloccia]

Did you subtract the dark cal from the flat field frame (i.e. the gain compensation frame?).  I think I'd suggested that earlier.  I think that's possibly what ID3 is for.  Try subtracting ID3 from ID4, and then use THAT as the gain compensation.  See if that improves the image even more.

[jadew]

@ John

ID3 is the image frame and besides ID4, all the other frames have big values for their pixels. In my camera's case, ID4 has values under 300, while all the other ones have values > 4000.

I think ID8 and ID7 can play a role in improving the image quality too, but I can't figure out how. Maybe they'll document these frames when they remove the SDK.

Edit:
I was wrong about the values, ID4 has values up to 4000+.

After a bit more fiddling around, I realized there's still a bunch of noise, but more subtle. It appears that either the gain or the offset drifts a lot, so even if you take a calibration frame to subtract from following frames, the noise will still resurface, fast.

On top of this noise there seems to be a lot of random noise too, so when the dynamic range is small, you get worse performance.

[John Coloccia]

I guess I'm confused with the frame numbering and such.  I'm looking at initial frame 3 and 4.  I guess I don't understand what the ID frames are.  Glad to see you have it mostly working, though.

[jadew]

Each frame has an ID that identifies it in its 10th pixel, which is one of the dead ones anyway (there's a pattern of dead pixels, by design).

The first frames have the following IDs:
frame 1: 4
frame 2: 9
frame 3: 8
frame 4: 7
frame 5: 10
frame 6: 5
frame 7: 1
frame 8: 3

After which, you'll periodically get ID 6 and then followed by ID 1 along with a shutter noise. These frames are similar and they serve as offset calibration.
After the offset calibration frames, you'll receive a number of ID 3 frames - actual images.

So, after the initialization sequence, you'll get something like this:
6, 1, 3, 3, 3, 6, 1, 3, 3, 3, 3, 3, 6, 1, 3, 3, 3, 3, 3, 3 etc.

Edit: Typo

Edit 2:
I think you're right. I'm only guessing but I think that those two frames can be used to derive the gain variation. From what I'm reading micorbolometer arrays don't have linear gain, which means that the gain has to be compensated differently based on the value being read.

[jadew]

Alright, I attached the executable. The source can be found at https://github.com/rzva/ThermalView.

It should compile on linux too.

You'll have to rename the files to ThermalView.zip.001 and ThermalView.zip.002 in order to unzip them - I couldn't upload them otherwise.

As you'll notice, the gain compensation is still a bit off, so if anyone can figure out why, I'd very much like to know.

Edit:
The files are compressed with the 7zip program.

Edit 2:
Single zip file: http://dumb.ro/files/ThermalView.zip

[jadew]

Here's a couple of screenshots.

PS: Sorry for making another post, they didn't fit in the previous one.

[miguelvp]

Thank you so much, I got them and will play with it soon after I'm done with a current project.

BTW, the images look great.

[jadew]

You're welcome, let me know how it goes.

I just did another fix that affects offset calibration correction. The updated binary is on that link, I removed the attachments.

One thing that's worth mentioning is that once it gets to a stable temperature, the pixels stop drifting that much and using an extra calibration frame yields steady results.

[frenky]

Hi.
I did some comparison of our efforts...

Original seek:


JoeC:


Miguelvp:


jadew:


frenky:


I am really sorry if I didn't correctly use any of your apps. I really tried to get the best results possible out of it.

Together we could get better results than Seek that for sure... But Seeks software has really improved in the last few months.

It would be great is if someone would focus on a better android app. The original one really needs a feature to set fixed temperature range.
If you move the camera the colors matching temps are changing which is really annoying. And you should be able to manually fit color palette into desired temp range.
That is something that I would be really excited about.

BR,
Frenky

[jadew]

@ frenky

Thank you for the comparison.

I was going to add a fixed span option to my program, but I'm still trying to figure out how temperature is calculated.

I suppose some of the values returned during the initialization routine would play a big role in this.