Flir E4 Thermal imaging camera teardown

[alank2]

I got a Flir E4 today.  There seems to be a vertical banding issue where you can see columns of variation that don't move when you move the camera.  Is that normal?

[Chanc3]

Not at all. Can you post an image of it?

[alank2]

It is bad.  Definitely going back.

[Chanc3]

Definitely something wrong with the detector. You'll need to return it under warranty if purchased new.

[alank2]

Thanks Chanc3 for your help.

[Chanc3]

No problem - the best thing with FLIR, is that all their detectors have a 10 year warranty (think the E4s are included).

[Solare]

Hi guys,

it seems you've talked me into purchasing this baby almost...

After reading hundreds of pages, I feel 'whew hoo' in my head and total mess

But still I have this question to ask,

Several times different users wondered, what actually E4 does with its 60 fps, but I failed to found answers in the thread...

So...

- It skips (decimates) actually taken but forbidden frames
- It averages the frames to reduce temporal noise

Does anybody really know this by this moment for sure?

Thank you!

[Fraser]

Well timed question. I only found out yesterday !

The FLIR cores that are limited to 9fps repeat the same frame several time over on the output of the Read Out (RO) electronics. In full frame rate mode every frame is passed to the output.

If the frame rate needs to be reduced by a factor of 'X' to meet ITAR then the RO electronics grab a frame, repeat it 'X' times at the output and then provides a new frame and the process continues. I thought that it was likely that frame layering was used to improve noise levels but I have seen no mention of such in the documentation that I read.

The output of the E4 core complies with the ITAR requirements and does not present the full frame rate at the RO electronics output. Changing the programming of the core frame repeat number will breach ITAR on this appliance.

Hope this answers your question.

Aurora

[Fraser]

The statement from FLIR on what they call slow frame rate cameras that do not require an export licence. The E4 is not mentioned by name but the same is likely to apply to it 

FLIR state:

"The reduced frame rate is achieved by replicating one frame four times for NTSC, or three times for PAL video. As standard NTSC is 30 frames per second, 30 divided by 4 = 7.5 frames per second. For PAL, 25 frames per second divided by 3 = 8.333 frames per second. We refer to this configuration as "slow video". Note that slow video frames are not averaged, rather, each frame is simply produced either three or four times in succession, depending on whether the camera is set for PAL or NTSC."

Aurora

[Solare]

Hi Aurora,
thanks for your reply!

so it seems, E4 appears to be a native 60 fps device, however most of the frames are replicated?! waste of energy!
Well done, Flir... now we know what Gods are doing to our brains in design time... decimating...

So...next question is obvious. Is the E4 hardware capable of pushing true 60 fps through the USB port. Let's say, I would be happy to have 60 fps recorded video!

There are some rather fast processes I would like to observe (t.ex, initial heat speading in radiator during ignition),
and in order for not to break any Laws I say, okay, I agree to see it as 9 fps realtime on screen, BUT have it in full fps recorded video! After all, being a Windows Embedded device with rather powerfull processor and memory, real-time encoding such a low resolution into *.mp4 is not a BIG DEAL even without hardware support.

do you think it is possible?
 
Thank you!

[Fraser]

In short, NO.

The read out electronics are part of the microbolometer so the image data is effectively 9 new frames per second from the data source. All other processing in the camera is working with that limited fps data.

Aurora

[Solare]

...arghhh,

things never fit ideal...

[mikeselectricstuff]

I can't say I'm totally convinced  that the E4 sensor doesn't output real 60fps to be avaraged by the FPGA to increase SNR, but it shouldn't be too hard to test using a moving target of a known speed, and looking at the motion blur.

Duplicated data would also require that the sensor have a full frame's worth of memory onboard - why would they go to the expense of that when they could just skip output frames? Or maybe there is some sort of analogue sample/hold going on.

I also wonder why the lens on the Ex0 is so much bigger...

 

[OrBy]

Well timed question. I only found out yesterday !

The FLIR cores that are limited to 9fps repeat the same frame several time over on the output of the Read Out (RO) electronics. In full frame rate mode every frame is passed to the output.

If the frame rate needs to be reduced by a factor of 'X' to meet ITAR then the RO electronics grab a frame, repeat it 'X' times at the output and then provides a new frame and the process continues. I thought that it was likely that frame layering was used to improve noise levels but I have seen no mention of such in the documentation that I read.

The output of the E4 core complies with the ITAR requirements and does not present the full frame rate at the RO electronics output. Changing the programming of the core frame repeat number will breach ITAR on this appliance.

Hope this answers your question.

Aurora

What defines the "core" in this case? Mike showed that the microbolometer itself was outputting at 60Hz, I poked around a bit and found there may be some limiting going on in the WinCE display driver itself. But makes the most sense to me that it would be done in the FPGA.

[Fraser]

In terms of ITAR it is the Microbolometer itself that is specifically regulated in terms of distribution of >9fps capable parts.

If FLIR are complying with ITAR regulations (failure to do so is serious in the USA) in the E4, then the microbolometer module itself must be self limiting to <9fps in a manner that an end user cannot change. Them's the rules. It can still produce a data stream of 30 or 60fps so long as the useful DATA is <9fps in terms of changing content.

In my FLIR PMxxx series the microbolometer is strictly controlled as it outputs 60fps, theoretically if I removed the microbolometer from the camera, the remaining chassis loses its ITAR liability, but in the real world just about everything concerned with the production of a 60fps thermal imager appears 'locked down' in terms of distribution across borders.

I have no reason to believe that the E4 microbolometer cannot achieve higher frame rates if FLIR wish to use it in regulated cameras. They usually state that their microbolometers are full frame rate (30 or 60fps) but are factory limited when sold for non ITAR regulated applications. I understand that it is a setting within the ROC but I cannot say how it may be changed as I do not have that information.

Aurora

[Solare]

Hi all,

looks I've triggered the topic

Really, big lenses... would make sense to lower distortion, ofc, but also as a reason to get higher singnal to noise ratio for the matrix of Ex0 which is not that good in terms of temporal noise?

Good point in frames replicating or skipping (either way, effectively decimating),
is that picture doesn't 'blur' upon movement along target... okay looks more like slide show, but not blurry...

I was surprised to know that this decimating is done right on the microbolometer scale... I would also use FPGA for this, that would allow for unifying process technology for microbolometers production for different fps devices, on the other hand, not so easy to re-program - this would satisfy the ITAR.

MIkle says it is 14 bit resolution, not 16... either way, if the datastream is recorded with some streamer or deep memory oscilloscope, and then run simple statistics (built-in) like histogram, or even calculate entropy divided by degree of fredom (60 in this case) when camera is slowly panning... that would answer the question quantitively.

[mikeselectricstuff]

MIkle says it is 14 bit resolution, not 16... either way, if the datastream is recorded with some streamer or deep memory oscilloscope, and then run simple statistics (built-in) like histogram, or even calculate entropy divided by degree of fredom (60 in this case) when camera is slowly panning... that would answer the question quantitively.

Data is definitely 14 bit.
Checking should be easier than that - just capture a few sequential frames and compare the same location in adjacent ones - noise will ensure some difference.
Of course if the duplication is analogue (sample/hold) then you may still see small differences.

[Solare]

I see... thanks for the useful info,

Hmmm.... regarding 14 bit data... That implies 14th bit is jittering because of noise, and hence 14 bit means 16384 levels, the whole temperature span is divided into (-20 to 250C = 270) / 16384 = 16 MKelvins. I remember some post where a user had some 'noise' parameter adjusted, and then he measured temporal noise to be almost 15 Mkelvins instead... from that, I would conclude that

1. E4 doesn't really change the temp span in either auto or locked or manually set boundaries? No need for this, 14 bits is enough.

2. E4's microbolometer is quite good in terms of temporal noise, maybe (just maybe) it would afford even higher resolution, if ADC would allow for.

But then, I remember another post, where user claims that E8 has some 'minimal tempurature span of 4 C' while E4 has only 8C minimal temperature span. No matter, how many levels are available.

Another restriction here?

In other words, do you know whether E4 really sets (reconfigure) temperature span in hardware at ADC level, or it does not?

This is actually important question, because if it does, the series of images may be further statistically processed to get rid of temporal noise convolved with ADC noise -> and get much higher temperature gradient detail, if the temperature span is set small... otherwise, this will only slightly help to improve sharpness

Thank you!

[Bud]

A doctored E4 has span 2C in manual scale mode.

[-jeffB]

How odd -- the narrowest manual range I can coax out of my hacked E4 is 4C (7.2 F). Wonder what's different between my configuration and yours?

[Bud]

I used whatever config janekivi published at the time.

[SeanB]

Hi all,

looks I've triggered the topic

Really, big lenses... would make sense to lower distortion, ofc, but also as a reason to get higher singnal to noise ratio for the matrix of Ex0 which is not that good in terms of temporal noise?

Good point in frames replicating or skipping (either way, effectively decimating),
is that picture doesn't 'blur' upon movement along target... okay looks more like slide show, but not blurry...

I was surprised to know that this decimating is done right on the microbolometer scale... I would also use FPGA for this, that would allow for unifying process technology for microbolometers production for different fps devices, on the other hand, not so easy to re-program - this would satisfy the ITAR.

MIkle says it is 14 bit resolution, not 16... either way, if the datastream is recorded with some streamer or deep memory oscilloscope, and then run simple statistics (built-in) like histogram, or even calculate entropy divided by degree of fredom (60 in this case) when camera is slowly panning... that would answer the question quantitively.

you do not need an extra framestore, you already have the one used to store the charge from the microbolometer elements under them, you just read it out according to the standard set for the viseo stream. I would hazard that there is a fuse bit on the die that is either blown or not blown to set the division ration of the system. Blown it is a 3 or 4 times division set by the standard in the input data from the microcontroller during initialisation, and if not blown it will set it to a full 60Hz and ignore the setup data there. Hard to unblow a fuse without destroying the die, and they can test all the dies and those that are functional but too slow at 60Hz will be downgraded to the 9FPS versions. Likely the fuse is hidden under the microbolometer array as well to prevent it from being bridged without destroying the array.

[Solare]

thanks guys,

okay...this picture is looking as a proof that manually set temperature span can be set down to 2C. However, as I can observe from the feedback, not all users are blessed with such the low span 

From this I conclude, that there is some setting which can be adjusted, which is default for the E8 (the latter case) to 4C and altered to 2C (the former case). Again, how it is called and where it can be adjusted? Can it be then adjusted to even smaller span? 1C? 0.5C? There are applications which would benefit greatly from such the low span setting.

Returning to the proof kindly provided, I must say that it proves nothing. If the whole span is -20C to 250C and the 14th bit is 15MKelvin (near noise), for 2C set span, we have ~7 bits (133 > 128 levels ) resolution, so this is again enough to see an unbanded image on such the low pixels count.

In other words, I still cannot see does the E4 hardware set the temperature span on its ADC or it doesn't.

But if Bud would be so kind to run simple statistics on the image, he would give us a good hint ... 7 bits are pretty visible on histogram (all the numbers falls into 128 bins not equally in count, but there are no numbers in between them - test with noise generator parameter set to 0!)

If, however, the ADC really sets its limits according manual setting, all the numbers will be different.

Also in this case thermal image entropy value calculated will be much higher than for the 7 bit image.

If the latter is true, we can de-convolute the tempral noise and much of the other types of noise by special procedures from statistical image processing, thus obtaining true, hi-res, low temperature span image and a branch of new applications for E4 arrives!

Regards,
Solare

[Fraser]

Hmmm not sure if I can help here but I can comment on FLIR professional cameras.

The PM series have 2 or 3 RANGES that are manually selected. After selection the camera runs a calibration routine which suggests that something significant in the image processing path has changed. ADC reference or whatever.

The E4 is a SINGLE range camera. It captures the full span data from the microbolometer and the image processing stages that follow select the data required for a specific span and centre temperature. It is just a moveable 'window' in a large span of temperature data.

The E4 does not recalibrate when spans are selected and I have no reason to believe that the ADC reference is changed. Greater temperature capabilities are also offered on pro cameras in the form of thermal filters. These are in the optical path just in front of the microbolometer. The E4 does not need such a filter as its maximum temperature is quite low and well within an ADC single range.

With regard to the minimum span of the E4. It was 8C on the E4 STD and 4C on the E8.
Way back in the early days of the E4 investigation it was realised that there was a noise generator used to degrade the images on the E4 AND there was a minimum SPAN specification in the configuration files. The E8 spec of 4C is sensible but the camera can be set to 2C minimum span if desired. Even professional cameras suffer image noise increase at very small spans. Less than 2C span can be challenging especially when you consider the camera tolerance specs.

Aurora

[Solare]

thanks Aurora,

Indeed, very useful information!

If I remember correctly from your earlier posts, you are one who played a lot with ZnSe lens... have to say, nice work!

I also consider buying ZnSe lens and experiment with close-up thermal imaging, however, I don't understand certain principles of operation in such the mode:

1. We have IR window correction seeting. Obviously, IR window will attenuate, so this attenuation must be compensated.
2. But in case of lens we have rather amplification of flux, not attenuation! That should affect the opposite way, but I cannot see how we can set camera for 'negative attenuation'?
3.What do you see with 50 mm lens on larger distances (several meter). Blurry? In other words, can the outter lens be adjusted so I an focus on distant targets as well, or I need a special telescopic lens assembly for this?