Flir E4 Thermal imaging camera teardown

[cynfab]

Just to confirm that the cam in RNDIS mode works fine on a Linux host with kernel 3.11 (I am using Linux Mint 13 with the kernel of the latest ubuntu version, package name "linux-image-generic-lts-saucy", version 3.11.0.14.13)

kernel messages when plugging the cam in:

[117540.997436] usb 1-6: reset high-speed USB device number 13 using ehci-pci
[117541.154362] rndis_host 1-6:1.0: RNDIS_MSG_QUERY(0x00010202) failed, -47
[117541.162965] rndis_host 1-6:1.0 eth1: register 'rndis_host' at usb-0000:00:1a.7-6, RNDIS device, 00:40:7f:xx:xx:xx

kernel version:

xxx@xxx01 ~ $ uname -a
Linux xxx01 3.11.0-14-generic #21~precise1-Ubuntu SMP Wed Nov 13 19:37:48 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

After plugging in the cam, the host fires up a new interface "eth1" with IP 192.168.0.1 and it's possible to connect to the cam via http/ftp/telnet
No problem at all  O0

bernroth, could you do a "cat /proc/sys/net/ipv4/conf/all/arp_accept"

I was using Mint 14 but with 3.5.0-17-generic kernel. and my arp_accept defaulted to "0" so the GARP packet that the camera sends, when plugged in, goes on deaf ears.

  tnx

[inline]

Happy Day, Flir E4 delivered by UPS just now and it's version 1.19.8  s/n: 63906xxx

Honestly with MSX it looks pretty darn great as is...can only imagine once I upgrade it. 

If it helps anyone, I placed my order on 12-4-2013 in the morning from Tequipment.net but due to issues at their warehouse shipping location in PA it didn't ship until yesterday, but I still received it as I'm really close to where they ship from.  So I get to play this weekend!! 

UPDATE:

OK, that didn't take long... Many thanks Mike for the great work and instructions.  Sweet upgrade! 

[bernroth]

bernroth, could you do a "cat /proc/sys/net/ipv4/conf/all/arp_accept"

arp_accept is 0 on my system.
I think they might have fixed the whole RNDIS issue somewhere between 3.8 and 3.11

If 3.11 available on your system?

Try:
aptitude search linux-generic-lts-saucy

If not, well, I would recommend to use Mint 13 as it's supported till april 2017 and has always all new kernels from the newest ubuntu distributions available.

[tomas123]

Incidentally could you also post the raw .gan and .crs files? That, and do you use exiftool for the gan files? I tried it some time ago but that didn't work. So in the interest of a quick fix I just ditch the 544 byte header and read the raw 16-bit data.

my png's are 16 bit raw values

simple rename .gan to .fff
start flir web viewer http://support.flir.com/webviewer (damned MS Silverlight) and in upload dialog type * and ENTER for select your *.fff file
now you can save the uploaded fff image as radiometric flir jpg image
use exiftool for expand 16 raw values from jpg

[mrflibble]

bernroth, could you do a "cat /proc/sys/net/ipv4/conf/all/arp_accept"

I was using Mint 14 but with 3.5.0-17-generic kernel. and my arp_accept defaulted to "0" so the GARP packet that the camera sends, when plugged in, goes on deaf ears.

FWIW, if have arp_accept set to 0. And when I plug in the E4 it now all works automagically. Just make sure you have the right udev rules (which you probably have), and set the ethx to allow-hotplug in your /etc/network/interfaces file. I used to add a static arp entry, but that's not really required so commented it out. Are you sure there's not some other curious setting blocking arp? When I do a ping, then I see the proper arp request + reply, followed by the icmp packets. Without needing that arp_accept.

Code: [Select]

23:29:15.434109 ARP, Request who-has 192.168.250.2 tell 192.168.250.1, length 28
23:29:15.435302 ARP, Reply 192.168.250.2 is-at 02:40:7f:0a:88:83, length 28
23:29:15.435314 IP 192.168.250.1 > 192.168.250.2: ICMP echo request, id 11272, seq 1, length 64
23:29:15.437701 IP 192.168.250.2 > 192.168.250.1: ICMP echo reply, id 11272, seq 1, length 64

Where 192.168.250.1 is my linux box, and 192.168.250.2 is the E4.

I just checked this after a cold boot of the E4. And yes, I cleared the arp cache before doing this, just to be sure.

[mrflibble]

Incidentally could you also post the raw .gan and .crs files? That, and do you use exiftool for the gan files? I tried it some time ago but that didn't work. So in the interest of a quick fix I just ditch the 544 byte header and read the raw 16-bit data.

my png's are 16 bit raw values

I have no problem using your png's, I just hoped maybe there was a feature in exiftool that I missed to directly read those .fff files. Because I already had tested the rename to .fff and then run exiftool on that. But all that tells you is that yes it's a FLIR file (based on the FFF signature in the header) and that's it.

simple rename .gan to .fff
start flir web viewer http://support.flir.com/webviewer (damned MS Silverlight) and in upload dialog type * and ENTER for select your *.fff file
now you can save the uploaded fff image as radiometric flir jpg image
use exiftool for expand 16 raw values from jpg

That is more work, not less. Guess I will just keep doing what I am doing. Oh look, 544 byte header. weeeey, *skip*.

[tomas123]

I just hoped maybe there was a feature in exiftool that I missed to directly read those .fff files.

ok, I wrote a simple config file for exiftool to read this special images

read this
http://www.workswell.cz/manuals/flir/hardware/A3xx_and_A6xx_models/Streaming_format_ThermoVision.pdf
for tag information ( FFF_TAGID_GainDeadMap = 5, FFF_TAGID_CoarseMap = 6)

using:

Code: [Select]

$ exiftool -config config.txt ds250C_we_ap_fi_le_static.gan -b -RawThermalImage > ds250C_we_ap_fi_le_static.tif

$ exiftool -config config.txt ds250C_we_ap_fi_le_static.gan
ExifTool Version Number         : 9.38
...
File Type                       : FLIR
MIME Type                       : application/unknown
Raw Thermal Image Width         : 320
Raw Thermal Image Height        : 240
Raw Thermal Image Type          : TIFF
Raw Thermal Image               : (Binary data 153804 bytes, use -b option to extract)



[tomas123]

some background infos

in pdf from last post you find the flir tag for "Raw Thermal Images"
    FFF_TAGID_Pixels = 1,

some flir cams save this 16 bit raw images as native byte sequence in reverse order

sample flir E40 (32 Byte Header)

Code: [Select]

$ exiftool IR_2316.jpg -v3 | grep -A4 'Tag 0x0001 ('
  | - Tag 0x0001 (38432 bytes):
  |     0ef4: 02 00 a0 00 78 00 00 00 00 00 00 00 9f 00 00 00 [....x...........]
  |     0f04: 77 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [w...............]
  |     0f14: bc 37 e9 37 ba 37 cd 37 ae 37 e1 37 ff 37 dd 37 [.7.7.7.7.7.7.7.7]
  |     0f24: f3 37 eb 37 c1 37 e2 37 c8 37 f2 37 05 38 e5 37 [.7.7.7.7.7.7.8.7]Phil Harvey (exiftool) decided to export the byte sequence in reverse order as tif image.

But some Flir Cams save the raw values as compressed 16 bit png
sample flir E4 (32 Byte Header)

Code: [Select]

$ exiftool FLIR0011.jpg -v3 | grep -A4 'Tag 0x0001 ('
  | - Tag 0x0001 (77714 bytes):
  |     0ef4: 02 00 40 01 f0 00 00 00 00 00 00 00 3f 01 00 00 [..@.........?...]
  |     0f04: ef 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [................]
  |     0f14: 89 50 4e 47 0d 0a 1a 0a 00 00 00 0d 49 48 44 52 [.PNG........IHDR]
  |     0f24: 00 00 01 40 00 00 00 f0 10 00 00 00 00 04 d6 3e [...@...........>]
see here for more informations about decoding flir headers
https://metacpan.org/source/EXIFTOOL/Image-ExifTool-9.25/lib/Image/ExifTool/FLIR.pm

[mrflibble]

Right, had some fun with FFTs just to see what we can see with regard to banding, vignetting, etc. Turns out that indeed you can see that in the spectral domain.

cal_norm.png


mrflibble-E4-spectral.png


mrflibble-E4-spectral_cutoff_000.png


mrflibble-E4-spectral_cutoff_060.png


mrflibble-E4-convolution_cutoff_060.png


The big spectral blobs near the X-axis in the 1st and 3rd quadrant look to be at about the right spot for the banding seen on the original cal pic.

What was new to me was the high frequency component on the Y-axis. That would seem to suggest that odd rows are slightly different from even rows. There is also a smaller high frequency component on the X-axis, which would indicate odd/even columns also being different but less so than the rows.

After I noticed this I added some stats to check for mismatches in the even/odd rows and columns.


Bad pixel threshold: 4 sdev
Bad pixel count: 62

                 |  MIN    MAX    MEAN      SDEV
-----------------+---------------------------------
Full image       |  32150  45489  36041.49  2372.99
Even rows        |  32397  45489  36123.39  2369.27
Odd rows         |  32150  45074  35959.60  2373.92
Even columns     |  32264  45489  36069.75  2379.06
Odd columns      |  32150  44516  36013.24  2366.61

Even-Odd mismatch for (ROW,COL): (+163.79 , +56.51)



What that says is that the average of the even rows is quite a bit higher than the average of the odd rows. And same story for the columns, but less so. Which indeed matches my suspicion that maybe the sensor pixel are either grouped by 2 or 4. Or maybe larger groups, haven't checked yet what's the most likely grouping based on stats.

And it also seems that the crappier the sensor the more pronounced the effect.

For example schdiewen's pretty darn awesome E4  looks to be far better in this regard.

schdiewen-E4-spectral.png



Bad pixel threshold: 4 sdev
Bad pixel count: 64

                 |  MIN    MAX    MEAN      SDEV
-----------------+---------------------------------
Full image       |  31863  45767  36190.67  2370.90
Even rows        |  32117  45408  36196.05  2374.12
Odd rows         |  31863  45767  36185.28  2367.69
Even columns     |  32117  45201  36208.34  2378.75
Odd columns      |  31863  45767  36173.00  2362.93

Even-Odd mismatch for (ROW,COL): (+10.78 , +35.34)


That looks to be one clean sensor... Note the lack of high frequency component. So unlike mine, this sensor looks to be fairly equal for the odd/even rows.

And tomas123's E40 is even better ...

tomas123-E40-spectral.png



Bad pixel threshold: 4 sdev
Bad pixel count: 17

                 |  MIN    MAX    MEAN      SDEV
-----------------+---------------------------------
Full image       |  32360  43359  36067.35  2222.82
Even rows        |  32360  43265  36065.73  2221.96
Odd rows         |  32422  43359  36068.98  2223.71
Even columns     |  32360  43265  36074.36  2226.46
Odd columns      |  32434  43359  36060.34  2219.19

Even-Odd mismatch for (ROW,COL): (-3.25 , +14.02)


And lucky for me, tomas123's E4 sensor looks to have about the same characteristics as mine. Both in terms of spectral components and also in terms of odd/even rows/cols mismatch. That always helps to verify you're not
just making stuff up. 

tomas123-E4-spectral.png



Bad pixel threshold: 4 sdev
Bad pixel count: 170

                 |  MIN    MAX    MEAN      SDEV
-----------------+---------------------------------
Full image       |  32144  47159  36220.38  2537.15
Even rows        |  32144  47159  36284.63  2530.87
Odd rows         |  32148  46964  36156.13  2541.83
Even columns     |  32200  47159  36262.19  2551.62
Odd columns      |  32144  47112  36178.56  2521.93

Even-Odd mismatch for (ROW,COL): (+128.50 , +83.63)


I'll just dump the rest in a zip so those interested can grab that, otherwise this post becomes one big picture attachement fest.

Update: well darnit. Zip file too big or something.

[mrflibble]

ok, I wrote a simple config file for exiftool to read this special images

read this
http://www.workswell.cz/manuals/flir/hardware/A3xx_and_A6xx_models/Streaming_format_ThermoVision.pdf
for tag information ( FFF_TAGID_GainDeadMap = 5, FFF_TAGID_CoarseMap = 6)

using:

Code: [Select]

$ exiftool -config config.txt ds250C_we_ap_fi_le_static.gan -b -RawThermalImage > ds250C_we_ap_fi_le_static.tif

$ exiftool -config config.txt ds250C_we_ap_fi_le_static.gan
ExifTool Version Number         : 9.38
...


Oh, nice! I'll test that right now.

Code: [Select]

exiftool -config exiftool_config_tomas123.txt ds250C_we_ap_fi_le_static.gan -b -RawThermalImage | convert - ds250C_we_ap_fi_le_static.png
exiftool -config exiftool_config_tomas123.txt ds250C_we_ap_fi_le_static.gan -b -RawThermalImage | display -

Update: works as advertised. Nice one.

[mikeselectricstuff]

What was new to me was the high frequency component on the Y-axis. That would seem to suggest that odd rows are slightly different from even rows. There is also a smaller high frequency component on the X-axis, which would indicate odd/even columns also being different but less so than the rows.

If you recall on the teardown, there are 2 high-speed datastreams - I wonder if there are 2 ADCs.
Though from memory I thought the interleaving was along the X axis, however it's plausible that some analogue elements are alternated on each side of the die, or maybe even the MEMs structure has some alternate-lineness. 

[unicorn5]

For example schdiewen's pretty darn awesome E4  looks to be far better in this regard.

So this may have already been said, but does it appear the sensors are graded, the crap goes to the E4, the quality the E8, and during manufacturing there maybe a rare situation where there's a lack of crap graded sensors so they use a better quality normally reserved for more expensive model. And so the difference between E8 and E4 for most people will actually be firmware AND HARDWARE?

[tomas123]

@mrflibble

Do you have also skew stripes:


Are the aberrations caused by the lens?

[mrflibble]

If you recall on the teardown, there are 2 high-speed datastreams - I wonder if there are 2 ADCs.
Though from memory I thought the interleaving was along the X axis, however it's plausible that some analogue elements are alternated on each side of the die, or maybe even the MEMs structure has some alternate-lineness.

That's a very good point. Two seperate DACs could definitely be at play here. For the odd/even rows this would actually already be enough. However I suspect there is also some alternate-lineness in the rows columns. If not only because of X-axis spectral content + column statistics, but also because the .crs file (correction) definitely shows some stripes.

[mrflibble]

@mrflibble

Do you have also skew stripes:


Are the aberrations caused by the lens?

Yeah, the stripes on my sensor are pretty close to what your E4 is showing. I was actually quite happy that that result rolled out after I made up some tests. That way it confirmed that at least one other sensor (+lens) out there shows similar behavior.

I think those aberrations are indeed caused by the lens. Actually it's probably a combination of both sensor and lens, but the "bendiness" can only be caused by the lens. A straight line could have been sensor. The trick is of course to model it and try and seperate the two by deconvolution, but that is for another day. 

Speaking of which ... actually a way to try and get a handle on that is the following:
- take several snapshots of a stable scene
- do the same, but now with the lens correction disabled in the image flow
- repeat a bunch of times + average the pics with, and the pics without correction
- do convolution of those two results
- profit!

In the past I've played around a little with enabling/disabling the various steps in the image flow, and you can indeed see the slight influence of the lens correction. So with that it's hopefully possible to seperate the two causes.

Edit: this is the one I mean ...

Code: [Select]

# disable lens correction
rset .image.flow.lens.partOfImgFlow false

# enable lens correction (default)
rset .image.flow.lens.partOfImgFlow true


[mrflibble]

So this may have already been said, but does it appear the sensors are graded, the crap goes to the E4, the quality the E8, and during manufacturing there maybe a rare situation where there's a lack of crap graded sensors so they use a better quality normally reserved for more expensive model. And so the difference between E8 and E4 for most people will actually be firmware AND HARDWARE?

Well, so far all we have is three E4's and one E40. I'd love some more .gan and .crs files for both E4 and other models as well so we can have a bigger comparison. As tomas123 pointed out, the lens on the E40 is also a lot better. And it has a pretty low bad pixel count too, so at least the E40 gets the nice goodies. I would assume the E8 would get better binned sensors than the E4 and about the same lens. If that is the case then on the E8 you'd get the same "bendiness" as on the E4, somewhat less bad pixels, and hopefully a less pronounced odd/even row effect. But who knows, need more data. 

[tomas123]

in this forum, no one will probably buy a E8

[mrflibble]

in this forum, no one will probably buy a E8

Indeed. Why buy an E8 when you can have an E8+. 

[TinyScooby]

Hi All-

I was registered with this username in the past, but seems it disappeared, maybe I wasn't active enough, not sure, but here I am again...

Anyways, I was wanted a FLIR camera for over a few years now, but cost for the high resolution was just too much.  You all got me to pull the trigger on this and I just ordered the E4 through Tequipment using the EEVblog discount (thanks!).  I am close to where they ship these from (they ship them out of PA, not NJ where their office is), so I should have it quick.  I'll report back on the firmware/calibration date, but I'm guessing I'll be good as the units they are shipping I was told they received last week and I think others reported the hackable firmware version as of last week.  I keep my fingers crossed until then... 

Is this discount still going on? If so, how does it work? thanks

[unicorn5]

Indeed. Why buy an E8 when you can have an E8+. 

Oh ofcourse but given that you guys know or suspect the sensors are bin'd with the trash going to E4 more appropriate title is E8-, as they can never be as good when their image quality is critically reviewed [discounting the lucky schdiewens of this word]
As you say more data needed & not many/any  E5,6,8 owners here.

[KingVidiot]

As promised here are some shots from the zoo.

By the way, either I'm already spoiled with the 320 res images or the hack didn't take since some of these don't look too smooth. Thoughts?

Videos coming soon...

[cuisinart]

Just a follow-up on my 8 degree gradient defective unit, they responded and rather promptly sent a replacement unit and an RMA shipping label for the first one.  Great service!  After seeing some of the image correction maps here, stored in the unit, I think the defective unit can probably be repaired pretty easily by fixing the aperture in place properly and re-doing the complete sensor calibration procedure.

[iceisfun]

223 stainless suppressor after ~15 rounds



50 bmg stainless suppressor ~7 rounds
individual baffles visible








Bird sleeping with head on back


Bird Suspicious about "Camera"


Unknown Revolver

[nova1200]

K.V. - yes, it looks like your E4 is still an E4. For example, here is the embedded thermal image from the African Gray parrot file:

[jlr134]

King...

Are the auto hot spot, auto cold spot and picture in picture functions present and working?