Flir E4 Thermal imaging camera teardown

[Taucher]

So FlashFS/system/appcore.d/config.d/E8.cfg doesn't work.

True and makes sense - as mentioned somewhere in this thread the config files are loaded by ascii order - uppercase letters come before lowercase letters - thus E will load before c(onfig) - but e will load after c The hack is just adding a second set of commands - which happen to be dupes of the existing config file .. of cause with nicer values

[Mordkanin]

Doesn't seem to be any issues with upgrading the 1.1. Happy owner of one, here.

[bronson]

Yes, but it's not so obvious when E8.cfg sorts below config.cfg in your ftp client. 

Wow, UVC mode and guvcview on Linux work fantastic.  Beautiful video.  Now I want a tripod mount so this thing can record my ham-handed hot air rework and welding attempts.

There appears to be a minor a problem with the triple RNDIS+MSD+UVC mode.  MSD doesn't appear to work -- the pictures aren't mounted.  MSD+UVC is fine, and MSD+RNDIS is fine too.  No big deal, just switch before plugging in.  It's definitely menu hack time -- pressing right for 10 seconds is getting old.

Examples next...

[bronson]

Here are a few pictures.  Some are pre-hack, some are post-hack.  It's obvious which are which.

Notice the framing inside the wall behind the xmas tree.  That wouldn't show up nearly as well without the hack.

Dog just came in from outside with cold ears, nose, tail, and toes.  And dog poop can't hide from me anymore.  Both pre-hack.  80x60 is still darned useful.

#0016: yum.

Finally, shoes a few minutes after taking them off.

[bronson]

Oh, and a dead pixel map if it's interesting.  Looks the same as everyone else's.

Mrflibble, you still want config data?  These files?

./FlashBFS/system/zeromap.fff
./FlashBFS/system/rotationmapcw.fff
./FlashBFS/system/rotationmapccw.fff
./FlashBFS/system/distmap.fff

[mrflibble]

Mrflibble, you still want config data?

Yes please. Specifically those four:

./FlashFS/system/maps/ds250C_we_ap_fi_le_static.gan
./FlashFS/system/maps/ds250C_we.crs
./FlashFS/system/maps/ds_we_ap_fi_leExFOL7_LCFMap.fff
./FlashFS/system/maps/ds_we_ap_fi_le_LCFMap.fff

./FlashBFS/system/zeromap.fff
./FlashBFS/system/rotationmapcw.fff
./FlashBFS/system/rotationmapccw.fff
./FlashBFS/system/distmap.fff

So far I haven't really looked at these last four .fff files. I think these are the same for all the E4 cameras. But it can't hurt to actually check that, so if you could include those as well that'd be great. All I know about those so far is that they are 468 16-bit value, and that the zeromap is indeed all zeros. I think they might be parameters for image processing blocks, but that is only a random quick guess. So far most of the playing around has been with the .gan (calibration image) and .crs (correction image).

[mikeselectricstuff]

There appears to be a minor a problem with the triple RNDIS+MSD+UVC mode.  MSD doesn't appear to work -- the pictures aren't mounted.  MSD+UVC is fine, and MSD+RNDIS is fine too.  No big deal, just switch before plugging in.  It's definitely menu hack time -- pressing right for 10 seconds is getting old.

I also found this, but at least on Windows it's not a problem as the Flir monitor thing implements a virtual drive on the toolbar via FTP/RNDIS, so only an issue if you need to plug into a different machine that doesn't have the RNDIS driver.

[mrflibble]

Those are really nice and interesting pictures. And thanks for sharing the numbers as well. After seeing my calibration pic the first time, I thought that the lower vignetting will lead to a lower dynamic range and thus my normalized picture would show more noise than yours. But now i know that our dynamic ranges are comparable, mine is even slightly larger.

So what's the "result", why are those pics that different, you have vignetting and banding, I have spatial noise instead? Perhaps the calibration pictures are already scaled?

Well to be honest there is no "result". What you see is what you get.  To have a meaningful "The Result [tm]" I'd really have to have more data across different cameras. Damned statistics, always needs more data. That said, even with limited data there are some interesting features that can be seen.

I took a look at the correction images recently. These are from your camera:

schdiewen-E4-CRS-spectral-nofilt.png

Clean.
schdiewen-E4-CRS-spectral-bandpass-10-60.png

Clean.
schdiewen-E4-CRS-spectral-highpass10.png

Hey look, high frequency components. And in the right locations to not just be figments of the imagination.

And as reference here's the GAN file again, but now a bit more properly processed. Bad pixels are now the average of the surrounding 4 pixels. When a surrounding pixel is bad, or this is right on the edge, then median value is used for the non-existing pixel. Mine didn't need this feature, but yours did due to bad pixels right on the edge.
schdiewen-E4-GAN-spectral-highpass10.png


Compare that to CRS and GAN file for my camera:

mrflibble-E4-CRS-spectral-nofilt.png

mrflibble-E4-CRS-spectral-bandpass-10-60.png

mrflibble-E4-CRS-spectral-highpass10.png

mrflibble-E4-GAN-spectral-bandpass10-60.png


I already posted about the spectral blobby things (technical term) in the 1st and 3rd quadrant. You can clearly see those on the last image. Those are related to the banding you see in the original calibration image. You can see those components all lie on a straight line through the origin, and that line is perpendicular to the bands (aka not making stuff up). And as you can see the correction image also contains those same spectral components, which is a nice confirmation. I'm not sure if it is easy enough to see those in a browser window, but if you open up that

Another thing to note about the correction images is that they are really really clean off-axis. Which is not so strange since those .crs files are the output of a correction algorithm. Which makes any off-axis components all the more interesting IMO.

[cynfab]

mrfibble,

For your data processing pleasure:

[OrBy]

More data for mrfibble to crunch attached.

[Mr-Beamer]

All your base data are belong to us.

[Fraser]

The E4 Map files from my camera.

I would be interested in what you can tell me about it.

[mrflibble]

Thanks a lot for all the data guys!  Keep 'm coming. 

I don't want to spam the thread with loads and loads of pictures (this time ), so for this post I will pick one sensor that shows some nice example of features. I will get to all the others as well, but I have to figure out a way to summarize things a bit.

Take this spectrum from a correction image.

OrBy-E4-CRS-spectral-highpass10.png


So that's the spectrum of the corrections Flir's calibration algorithm spits out, while they try to find a good tradeoff for this particular camera. You can see the two regions along a roughly 45 degree line. Just by looking at that you can see that "ah, there's probably something that is going to cause banding, and those bands will be perpendicular to that ~ 45 degree line.

Alright, lets look at the spectrum of the calibration image.

OrBy-E4-GAN-spectral-highpass10.png


Yup, definitely stuff along that line. In the correction image (CRS) it was mainly centered around one frequency, but in the calibration image (GAN) you can see that it is smearing some spectral power along the line. Which means ... feature is in the same direction, but the frequency magnitude (aka line width) is varying over a certain range.

OrBy-E4-GAN-normalized.png

Hey look, isn't that what we just described?  The banding is indeed perpendicular to that line in the spectral image, and you see thin bands (high frequency) near the edge of the image and wide bands (low frequency) near the center of the image. The cause of all that being the lens I'd think.

Other stats from OrBy's E4:

Bad pixel threshold: 4 sdev
Bad pixel count: 81

Stripsize = 1    |  MIN    MAX    MEAN      SDEV
-----------------+---------------------------------
Full image       |  31855  45060  35748.66  2408.95
Even rows        |  31855  45007  35781.32  2397.23
Odd rows         |  31972  45060  35716.00  2420.21
Even columns     |  31937  45060  35774.87  2411.03
Odd columns      |  31855  45007  35722.44  2406.62

Even-Odd mismatch for (ROW,COL): ABS(+65.32 , +52.43)  REL*1000(+1.82714 , +1.46655)

It does show odd/even mismatches, more than some cameras, less than others. By having looked at the numbers for several cameras now I'm reasonably sure that there is an underlying regular pattern to the pixels on the die.

And here is the same thing run against the correction image, after an arbitrary scale factor to get numbers that fit the columns nicer. The relative numbers (which is what we're after) do not change when scaling.

Stripsize = 1    |  MIN    MAX    MEAN      SDEV
-----------------+---------------------------------
Full image       |  21000  43000  33171.28  1090.85
Even rows        |  21000  43000  33264.19  1086.16
Odd rows         |  29000  43000  33078.36  1087.62
Even columns     |  29000  43000  33456.93  1049.45
Odd columns      |  21000  43000  32885.62  1056.12

Even-Odd mismatch for (ROW,COL): ABS(+185.83 , +571.30)  REL*1000(+5.60224 , +17.22279)

Which says that yes there is quite a bit of alternate line-ness and more so for colums. And when just eyeballing the normalized correction image you can also see definite stripes. Well, either that or I have been staring at this stuff for too long. 

OrBy-E4-CRS-normalized.png


And for the sake of completeness the bad pixel map:
OrBy-E4-GAN-bad_pixels.png

[Fraser]

Excellent analysis of the data, and fascinating reading for me.

I am still trying to understand the meaning of it all, but will keep reading until I do 

[Fraser]

For those interested in the performance of the lens in the E4, I suggest that you read the attached document that may also be found here:

http://www.lightpath.com/literature/technicalPapers/MoldedChalcogenideGlassTechnology.pdf

Section 3 details lens performance considerations.

 

[Taucher]

Btw guys - this seems to be getting the largest topic in EEVblog history - just the OWON thread has more replies (for now) 
... and that one started back in 2011 ... 

Edit: added data:
REVIEW - Owon SDS7102 - A look at the SDS series from Owon New
2308 Replies
246620 Views

Flir E4 Thermal imaging camera teardown
2222 Replies
72636 Views

Hantek - Tekway - DSO hack - get 200MHz bw for free New
2135 Replies
437648 Views

Sniffing the Rigol's internal I2C bus New
1926 Replies
141373 Views

REVIEW - Rigol DS2072 - First Impressions of the DS2000 series from Rigol New
1827 Replies
143152 Views

[OrBy]

Excellent analysis of the data, and fascinating reading for me.

I am still trying to understand the meaning of it all, but will keep reading until I do 

I am with Aurora on this one on both counts

[jlr134]

I just received a new E4 ....(E4 1.0, Software 1.19......... My problem is that I load the new configuration into the camera using FLIRInstallNet and the original hack (pg.33) the program says it loaded the files without problem .I then powercycle the camera but the new configuration dosnt stay in the camera .This is the second camera Ive done, first one went fine( after you guys straightened me out) and Im doing the process exactly like the first.  I was hoping someone could give me an idea  as to what the problem could be (  Ive triple checked the CRC01)

[Taucher]

I just received a new E4 ....(E4 1.0, Software 1.19......... My problem is that I load the new configuration into the camera using FLIRInstallNet and the original hack (pg.33) the program says it loaded the files without problem .I then powercycle the camera but the new configuration dosnt stay in the camera .This is the second camera Ive done, first one went fine( after you guys straightened me out) and Im doing the process exactly like the first.  I was hoping someone could give me an idea  as to what the problem could be (  Ive triple checked the CRC01)

Check out EzCRC (see my footer), verify that you did enter your serial number in the config file, ensure e8 is lowercase and that the path does not contain a surplus dirname or lacks something.
Alterantive: put the cam into RNDIS mode and upload the file by FTP (see some postings above)
PS: Powercycle = remove battery = hard reboot!

[schdiewen]

After dis- and reassembling my E4 to clean my dusty sensor, the visual and thermal images were shifted against each other, making MSX useless for me. This problem is now solved and here comes a short howto, in case someone else faces the same challenge.

Warning: It involves messing with the calibration file unique to your cam, so 1) know what you're doing and 2) have a backup!

Taucher posted some registers which can be modified by rset:
https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg327178/#msg327178

This way you could realign your MSX image by adjusting xpanVal and ypanVal, unfortunately I was not able to make it persistent. But since the parallax error depends on the distance to the object to be imaged, it wouldn't have helped anyway.

For a given userDistance z the shift d ((x,y)panVal = floor(d)) between visual and thermal image is calculated by

Code: [Select]

d(z) = c1 + c0/z                        (1)
with c0 and c1 being constant values that are factory calibrated into your cam. d, c0 and c1 are 2-dimensional (x,y), units are [px] for d and c1, [m] for z and [px*m] for c0. The coordinate system starts with (0,0) in the upper left corner, d is the shift of the visual image relative to the thermal image (see below for example).

The constants c0 and c1 are provided by \FlashFS\system\calib.rsc, here are mine:

Code: [Select]

...
.calib.visual.fusion.leExFOL7.C0X double -0
.calib.visual.fusion.leExFOL7.C0Y double 8.1835451
.calib.visual.fusion.leExFOL7.C1X double -8.0483421
.calib.visual.fusion.leExFOL7.C1Y double -2.2975248
...
There are more parameters important for MSX (zoom, rotation, ...), but I didn't need to touch them. So I'm only covering horizontal and vertical shift here.

You have at least two options to calculate new constants for your calibration file:

1) You could set these four parameters to zero, copy the resulting file with a new CRC32 back to the cam, overwriting the original one (keep a backup!) and reboot the E4. To recalculate new values for c0 and c1 you need to get at least two MSX images A and B with some distance between them, determine d_A(z_A) as well as d_B(z_B) and solve the system of linear equations to:

Code: [Select]

c0 = (dA - dB)/(1/zA - 1/zB)            (2)
c1 = (dA*zA - dB*zB)/(zA - zB)          (3)
Hint for the sign of d: if your visual image appears 5px to the top and 3px to the left of your thermal image, then d = (-3px, -5px).
Note that you don't need to configure z in the cams settings for this, with c0=c1=(0,0) while measuring, (x,y)panVal will be 0 for all z as well.

2) I found it easier to use rset to modify xpanVal and ypanVal (see Tauchers post) on my connected E4, until my MSX images looked the way I wanted. You may want to temporarily disable auto calibration while doing this or wait for the cam to settle.
I started with a very distant object, in my case it was a street light about 50m away, a building front with windows will also work. Now you already have c1, because c1 = d for large z. Then you need a second image of a nearby object, it should be small and centered in your image to minimize error. I used my doors peep hole at about 25cm away and adjusted (x,y)panVal and the distance to get a proper fit. Now you can use equation (1) to get c0.

My new calibration file obtained by 2) worked great, at least indoors, distant objects were still off by 3px. Reason: the configurable userDistance in the cams settings ends with ">3" (cam uses z=3m then), which makes it impossible to reach c1 for infinity. I decided to overcorrect my parameters and used my obtained c1 as if it were d_A at z_A=3m and calculated new constants with (2) and (3). Now I have perfect MSX even for great distances, while indoors my actual z slightly differs from the configured userDistance. However, I think it would be a better solution to have additional menue entries for greater distances (e.g. ">10").

Another thing worth mentioning: The cam doesn't round() the result for (x,y)panVal, it just chops off everything following the decimal point. So if your dy results to 5.99px at z=3m you'll end up with a 5px shift being used. You may want to adjust the constants to avoid this and could use rls .image.fusion to see the actual values the cam is using.

Some additional information about MSX can be found in patent EP2634747:
https://data.epo.org/publication-server/rest/v1.0/publication-dates/20130904/patents/EP2634747NWA1/document.pdf

[jlr134]

Thank you Taucher. I remember seeing your EZCRC01 back around page 100 but my computer dosnt want to run it , I sure did ( not valid win 32 message).
Anyway I considered alll your suggestions and the only one I couldn't  be %100 sure of is the CRC01( and I ran it 3 times) so I tried it again, got the right number
and now I have two cameras . Much gratitude to you and all the other heavy thinkers that hang around here. Ive read every post at least once and if I understood %30
of what I read Id be lucky(but I can try for %35)

thanks

[Taucher]

Thank you Taucher. I remember seeing your EZCRC01 back around page 100 but my computer dosnt want to run it , I sure did ( not valid win 32 message).
Anyway I considered alll your suggestions and the only one I couldn't  be %100 sure of is the CRC01( and I ran it 3 times) so I tried it again, got the right number
and now I have two cameras . Much gratitude to you and all the other heavy thinkers that hang around here. Ive read every post at least once and if I understood %30
of what I read Id be lucky(but I can try for %35) thanks

You're welcome, glad that it finally resolved

@Win32 error - did you try the -static version - if yes - which windows version, how many bits, ran from .zip or extracted the archive first?
Steps to reproduce the error?

[tomas123]

@schdiewen

great work.

some informations to the same topic:

I spend some time in self made MSX images for a  Flir E40 (old version of E40 without msx).
On E40 the axis of real and thermal cams are shifted. By focussing the thermal image the real image moves by a diagonal line...

In this worth reading thread I asked for a solution for finding the best overlapping between real and thermal image with imagemagick:
http://studio.imagemagick.org/discourse-server/viewtopic.php?t=23318&p=97895

I reversed engineered the exif tags from flir jpg (thanks to Phil Harvey).
The calibrated position coordinates (offset) from thermal to real images are also inside Flir-PiP-Images
see my old post: http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,4898.msg24156.html#msg24156

after then I worked out two imagemagick overlay filters for a MSX simulations and a nice amboss effect
here you find a sample (see the php script)
https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg342072/#msg342072

MSX


and amboss (see structure of table plate)


PS: Flir has a patent for a simple image processing procedure?

[Fraser]

From what I understand the differenece between FLIR MSX and previous forms of image fusion is the edge capture that produces  boundary lines of objects and detail rather than just a whole visual image merged into a thermal image which can obscure thermal detail.

[tomas123]

exactly, a simple Photoshop / Gimp / ImageMagick image calculation