OpenIRV. ISC0901B0 (Autoliv NV3, FLIR E4/5/6/8) based opensource thermal camera

[Bill W]

Wouldn't it have to be a pretty terrible lens to not be able to resolve  336x256, for example?

Not at all.  The real issue is the lens 'circle of confusion' versus the pixel pitch.  Lots of appropriate maths at
https://www.dofmaster.com/dofjs.html

Our thermal lenses score badly due to the large apertures used and the expense of processing or multiple elements.
A simple 2 element 'spherical' design (50mm f/0.7) meant for a Pevicon tube really does not resolve well on 17µm pitch.

Bill

[bap2703]

Do you have any ideas how to check this lens capabilities?

At first you can probably do a simple check:
Can you get a sharp image of a point or edge source?

The real thing is called measuring the modulation transfer function (MTF) of the lens.
Basically you describe an image in terms of spatial frequencies: how fast the luminosity is changing quickly across pixels.
Details equal to high frequencies for example. Since you played with frequency domain filtering of images you'll quickly grasp how that works.
Think of lenses like low-pass filters: in order to get more details, you need to grab more high-frequencies.
Lens designers would probably match the cut-off to the targeted sensor pitch if it allows to build it cheaper.

You can probably get to half the pitch without too much trouble (or software sharpening), but more than that you'll probably notice the lens limitations.

You can read more there:
https://www.umicore.com/en/newsroom/the-evolution-of-lens-designs-for-12micron-uncooled-lwir-detectors/

[_Wim_]

I would give these a try: ...
180µm for 100V, should be around 18µm for 10V, and as you are 3D printing your housing and absolute accuracy is not required for this application, I think these would be sufficient.

Hm... the characteristics look quite fantastic for this dimentions. Have you ever used this parts?

For example, both different well known manufacturers, and the parts with common dimentions have commom characteristics:
Kemet's AE0203D18H18DF from here: https://ru.mouser.com/datasheet/2/212/1/KEM_P0101_AE-1518874.pdf
PI's P-882.51 from here: https://static.pi-usa.us/fileadmin/user_upload/physik_instrumente/files/datasheets/P-882-Datasheet.pdf

I have absolutely no experience with piezo actuators, but chinese ones look really strange. Maybe I'm wrong, not sure this is truth. We need anyone how can prove that.

I have no experience with those yet. I did an experiment with a low cost piezo buzzer (with the intention of building an Scanning Fabry-Perot Interferometer like http://repairfaq.cis.upenn.edu/Misc/sale/sfpiins1.htm).

This is the type of piezo I am talking about: https://nl.aliexpress.com/item/4000120679339.html?spm=a2g0o.productlist.0.0.5cde25c3H0l9jK&algo_pvid=39a8075f-6e25-492a-911d-34217e99d7f2&algo_expid=39a8075f-6e25-492a-911d-34217e99d7f2-3&btsid=0bb0623416055593233587628ed6a4&ws_ab_test=searchweb0_0,searchweb201602_,searchweb201603_

These have a ceramic layer of only approximately 0.2mm thick. It was connected directly to my signal generator with a 10Vpp output. A mirror was glued to the piezo, and distance was measured with a DIY interferometer (details here: http://www.repairfaq.org/sam/uMD1/). This gave me a little less than 250nm pp. Taking into account the ceramic material was only 0.2mm thick, this would mean 6.25µm pp for a 5mm thick material. This is below the 18µm spec for 10V, but if similar specs, with a higher voltage, 17µm must be easily achievable.

To be sure, I also ordered a set (https://nl.aliexpress.com/item/32952294614.html?spm=a2g0o.productlist.0.0.4da739052Pa8qc&algo_pvid=aa231da8-f9e6-4c2d-97a4-be556ee5db3d&algo_expid=aa231da8-f9e6-4c2d-97a4-be556ee5db3d-0&btsid=0bb0624516055606928738744e86ab&ws_ab_test=searchweb0_0,searchweb201602_,searchweb201603_) and will test them the same way.
 

[_Wim_]

The results above need to be divided by 2, as I used a plane mirror interferometer setup instead of my "usual" linear setup with retro reflector, but did not change the settings accordingly in µmd1.

So this means with a 10V signal that kind of piezo only does 125nm instead of the 250nm shown in the result plot.

[LesioQ]

The ultimate limit will be Airy disk dia, i.e. physics.

http://www.calctool.org/CALC/phys/optics/spot_size

[VGN]

At first you can probably do a simple check:
Can you get a sharp image of a point or edge source?

Sure.

Experiment #1:
The distance to the object is about 5m. I used an aluminium tape to make a sharp edge. I was playing with focus and camera position for ~20 minutes. This is the best edge sharpness, that I could achieve. I started to feel lens limitations)

[VGN]

At first you can probably do a simple check:
Can you get a sharp image of a point or edge source?

Experiment #2:
The distance to the object is about 0,4m:

[VGN]

The real thing is called measuring the modulation transfer function (MTF) of the lens.
Basically you describe an image in terms of spatial frequencies: how fast the luminosity is changing quickly across pixels.
Details equal to high frequencies for example. Since you played with frequency domain filtering of images you'll quickly grasp how that works.
Think of lenses like low-pass filters: in order to get more details, you need to grab more high-frequencies.
Lens designers would probably match the cut-off to the targeted sensor pitch if it allows to build it cheaper.

bap2703, thank you very much, that was really helpful! Looks like I started somehow understand what is going on... 
So, it turn out that using pixel shifting I just actually decrease the sensor pixel pitch. And if I want to keep the image sharpness, I should increase the lens "bandwidth" (as it is an optical low-pass filter)

From the article:
Since the lens performance is essentially diffraction limited, there is only one way to increase the MTF of the lens so that it remains constant for the higher spatial frequency. That is to make the lens faster in the ratio of the change in pixel pitch.

Ok, so if I got it right, we need a faster lens if we want to decrease the pixel pitch. Probably, first we should determine the lens f-number.

You can probably get to half the pitch without too much trouble (or software sharpening), but more than that you'll probably notice the lens limitations.

If I'm not wrong, not only lens limitations. As the pixel itself is not a spot, i.e. it has some dimentions, we will also face with pixel overlap at very small steps.

[VGN]

I have no experience with those yet. I did an experiment with a low cost piezo buzzer (with the intention of building an Scanning Fabry-Perot Interferometer like http://repairfaq.cis.upenn.edu/Misc/sale/sfpiins1.htm).

This is the type of piezo I am talking about: https://nl.aliexpress.com/item/4000120679339.html?spm=a2g0o.productlist.0.0.5cde25c3H0l9jK&algo_pvid=39a8075f-6e25-492a-911d-34217e99d7f2&algo_expid=39a8075f-6e25-492a-911d-34217e99d7f2-3&btsid=0bb0623416055593233587628ed6a4&ws_ab_test=searchweb0_0,searchweb201602_,searchweb201603_

These have a ceramic layer of only approximately 0.2mm thick. It was connected directly to my signal generator with a 10Vpp output. A mirror was glued to the piezo, and distance was measured with a DIY interferometer (details here: http://www.repairfaq.org/sam/uMD1/). This gave me a little less than 250nm pp. Taking into account the ceramic material was only 0.2mm thick, this would mean 6.25µm pp for a 5mm thick material. This is below the 18µm spec for 10V, but if similar specs, with a higher voltage, 17µm must be easily achievable.

To be sure, I also ordered a set (https://nl.aliexpress.com/item/32952294614.html?spm=a2g0o.productlist.0.0.4da739052Pa8qc&algo_pvid=aa231da8-f9e6-4c2d-97a4-be556ee5db3d&algo_expid=aa231da8-f9e6-4c2d-97a4-be556ee5db3d-0&btsid=0bb0624516055606928738744e86ab&ws_ab_test=searchweb0_0,searchweb201602_,searchweb201603_) and will test them the same way.

Nice project!

This gave me a little less than 250nm pp. Taking into account the ceramic material was only 0.2mm thick, this would mean 6.25µm pp for a 5mm thick material. This is below the 18µm spec for 10V, but if similar specs, with a higher voltage, 17µm must be easily achievable.

Agree, that makes sense.


Found this "datasheet" for AL1.65X1.65X5D-4F, according to it only 3,8um@90V
https://img.alicdn.com/imgextra/i2/71977092/TB2wz4ybNaK.eBjSZFAXXczFXXa_!!71977092.png

To be sure, I also ordered a set and will test them the same way.

Anyway, I will be very grateful if you share your results with us!

[Fraser]

Just for information, the Autoliv camera is basically a variation on the TAU and TAU 2. Specifications for the various lens options are available. In my experience modern lenses for the TAU cameras are all of adequate resolution to work well with a VGA 17um microbolometer. If you look at suppliers, such as Ophir, it is clear that modern thermal imaging lenses are more than capable of VGA resolutions on 17um pixels and even XGA on 12um pixels.

I would be very surprised if the lens installed in a TAU, or TAU based core was not capable of at least VGA on 17um pixels. Lens manufacturing has become finely honed to offer affordable thermal imaging lenses capable of good resolution. Even Chalcogenide IR Glass lenses can achieve such good performance these days provided the lens designer does his job correctly. A manufacturer will normally design a lens set that will work with a large range of microbolometers and the Ophir catalogue shows this. XGA 12um capability is now common. I attach a link to the TAU brochure and the Ophir catalogue for information. How I would love to have free access to the Ophir range of lenses  . Take a look at the 65221 19mm lens specification on page 52. That is not a super expensive lens yet it is adequate resolution for 1024 x 768 12um pixels. That 19mm lens is used in the ThermApp camera.

http://52ebad10ee97eea25d5e-d7d40819259e7d3022d9ad53e3694148.r84.cf3.rackcdn.com/UK_Z24-FLIR_Tau2_Family_Brochure_DS_CAT-08.pdf

http://52ebad10ee97eea25d5e-d7d40819259e7d3022d9ad53e3694148.r84.cf3.rackcdn.com/UK_Z24-FLIR_Tau2_Family_Brochure_DS_CAT-08.pdf

From my perspective, I just cannot see a modern thermal imaging lens manufacturer going to the effort of making a poorly performing lens that cannot cope with VGA 17um pixels as a minimum. Where the mechanical limits permit, I would expect most modern lenses to cope well with XGA. It is true, as Bill_W has stated, that older thermal imaging cameras that used much larger pixels and arrays that were QVGA or smaller did have resolution limits associated with the lens block. That was an era when pure Germanium lenses were cut on mono point diamond lens forming lathes. Such was a very expensive process and remains so to this day. The lenses tended to be cut to the requirement rather than to needlessly exceed it at additional cost. Times have changed with the introduction of Chalcogenide IR glass lenses that are precision moulded and more advanced pure Germanium cutting processes. Thermal camera lens manufacturing has had to advance to keep in step with newer, smaller and higher resolution imaging sensors.

I caveat all of the above by saying that a manufacturer who wants a camera built down to a certain production cost may still cut corners on the lens part of the design. It is still possible to have relatively poorly performing thermal imaging lenses produced where image quality is not a high priority. Such lenses are commonly made for less demanding applications such as industrial IR temperature sensors so lens manufacturers do have relatively low grade lens production facilities.

FLIR went as far as using a ‘printed’ diffractive silicon lens in their Lepton core and that lens is resolution limited. How I wish there was a conventional lens option for that core.

Fraser

[bap2703]

Experiment #2:
The distance to the object is about 0,4m:

I'd say you lens looks definitely adequate to take advantage of superresolution.

If I'm not wrong, not only lens limitations. As the pixel itself is not a spot, i.e. it has some dimentions, we will also face with pixel overlap at very small steps.

You are right and that's mathematically all the same:
- the response of the lens is an image where every source point is spread (and that's called the PSF, point spread function)
- a pixel has a spatial response too, be it from it's physical size or some "antennae" effect (you posted picture of pixels with a grid structure for example)

When you know them, you can to some extent compensate their effects.
That's called deconvolution and it's used everywhere.

[_Wim_]

Found this "datasheet" for AL1.65X1.65X5D-4F, according to it only 3,8um@90V
https://img.alicdn.com/imgextra/i2/71977092/TB2wz4ybNaK.eBjSZFAXXczFXXa_!!71977092.png

  ? That could be correct also of coarse! Anyway, it will be fun to play with, and I need to "use" my interferometer...

Found this "datasheet" for AL1.65X1.65X5D-4F, according to it only 3,8um@90V
https://img.alicdn.com/imgextra/i2/71977092/TB2wz4ybNaK.eBjSZFAXXczFXXa_!!71977092.png
Anyway, I will be very grateful if you share your results with us!

 
Will do, but as we all now, it can take a while until the Chinese Santa Clause comes to our door...

[VGN]

I'd say you lens looks definitely adequate to take advantage of superresolution.

I hope that very much too. But I also want to try to prove this theoretically if it is possible.

The ultimate limit will be Airy disk dia, i.e. physics.

http://www.calctool.org/CALC/phys/optics/spot_size

Ok, lets calculate this Airy disk dia.

First, let's get the f-number of the lens.
According to this datasheet (table 6.1): http://www.safetyvision.com/sites/safetyvision.com/files/FLIR_PathFindIRII_User_Guide_1.pdf
The lens focal length is f = 19mm (seems to be true). The aperture circle dia D = 15mm. In this way f-number = f/D = 19/15 = 1.26(6). The closest standard value is 1.25.
So, let's say that we have a f/1.25 lens.

Finally for LWIR spectral band of (8 - 14um) we have an Airy disk dia of (21.35 - 42.7um). Well...even smallest value of this range is much larger than the actual pixel size for a 17um pitch technology...

Again, I'm not good enough with optics, but the result looks a bit weird for me, I didn't expect such values. Any ideas what am I doing or interpreting wrong? 

[Fraser]

Your calculations look correct to me.

I did not include the link to the Ophir catalogue in my earlier post, sorry.

Look at the 19mm lens on page 52 and enter its specs into an on line Airy Disk calculator (as I did). It verifies your calculated result.

https://applied-infrared.com.au/images/pdf/Ophir_Thermal_Imaging_Lenses_Catalog.pdf

I believe you are showing the limitations of the optics used in thermal imaging when small pixel sizes are involved. This may go some way to explain why microbolometer arrays that use 12um pixels struggle with contrast.

I believe anti-aliasing and dynamic contrast enhancement techniques are employed to improve the contrast in images produced by thermal cameras. It would appear that any core that uses a pixel size smaller than 25um is battling the Airy Disk limits of the commonly available optical systems.

Thank you for highlighting this situation as I have not previously looked at it for thermal lenses 

Airy Disk Calculator :

http://www.calctool.org/CALC/phys/optics/f_NA

For the referenced 19mm Ophir lens the Airy disk is 27um at 10um wavelength.

Fraser

[bap2703]

The game doesn't end with calculating the airy disc diameter ! That's a common mistake.

What do you want to do next?
--> sample it at more than just one pixel per airy diameter!
Because you're not really interested in sampling the airy disc with one pixel, but sampling the tiny contrast (in amplitude AND space) resulting from merging two close diffracted spots.

You can sample at a pitch much less than the airy diameter.

I found that document that appears to explains it quite well (I had a quick glimpse at it but it seems good).
https://www.researchgate.net/publication/336775082_Resolving_some_spatial_resolution_issues_-_Part_2_When_diffraction_takes_over

That one says a pitch of 1/6 of the airy diameter !

[VGN]

Thank you for highlighting this situation as I have not previously looked at it for thermal lenses 

Thanks to LesioQ and our cruel, but beautiful universe! 

The game doesn't end with calculating the airy disc diameter ! That's a common mistake.

What do you want to do next?
--> sample it at more than just one pixel per airy diameter!
Because you're not really interested in sampling the airy disc with one pixel, but sampling the tiny contrast (in amplitude AND space) resulting from merging two close diffracted spots.

You can sample at a pitch much less than the airy diameter.

I found that document that appears to explains it quite well (I had a quick glimpse at it but it seems good).

First, thank you for this article, it is really mind changing. I recommend everyone to read it.
Second, you are absolutely right, we can sample at a small pitch to get maximum possible spatial resolution that current lens is capable to provide. I found quite nice picture from one astronomy forum, that shows the advantage of high accurate airy disk sampling.

That one says a pitch of 1/6 of the airy diameter !

Here is a quote from the article, explaining why we are talking about 1/6 of the airy diameter for those, who didn't find it:

Since we know from the Rayleigh criterion that Δxmin equals the radius of the Airy disc, it follows that the best
‘match’ between sensor characteristics and the optics is achieved when the detector pitch p equals one‐third of
the Airy disc radius. In other words: three pixels suffice to resolve the Airy disc radius entirely. The reasoning
then goes that above that threshold, the optics diffraction limitation kicks in. So digitising the radius of the PSF’s
central part with more than three pixels does not improve the final spatial resolution of the image because the
detector only resolves diffraction blur at that point.

BTW, I didn't now, but FLIR has a special UltraMax technology, that allows to increase the resolution. It is very silimilar to Fluke's superresolution.
The most interesting moment is at 0:51, the main idea is the same, just sample the data in gaps between pixels, as active pixel area is really small.
Full video: https://youtu.be/8y2lV3nIhvc

[VGN]

Some updates.

Today received new two layer PCBs, i.e. buttons, display and the magnets holding ring! Going to assembly this boards today.

Unfortunately looks like due to covid-disaster manufacturer delays boards release, that's why main and peripheral boards will apear in my hands only at next week.

[Max Planck]

I'd say you lens looks definitely adequate to take advantage of superresolution.

I hope that very much too. But I also want to try to prove this theoretically if it is possible.

The ultimate limit will be Airy disk dia, i.e. physics.

http://www.calctool.org/CALC/phys/optics/spot_size

Ok, lets calculate this Airy disk dia.

First, let's get the f-number of the lens.
According to this datasheet (table 6.1): http://www.safetyvision.com/sites/safetyvision.com/files/FLIR_PathFindIRII_User_Guide_1.pdf
The lens focal length is f = 19mm (seems to be true). The aperture circle dia D = 15mm. In this way f-number = f/D = 19/15 = 1.26(6). The closest standard value is 1.25.
So, let's say that we have a f/1.25 lens.

Finally for LWIR spectral band of (8 - 14um) we have an Airy disk dia of (21.35 - 42.7um). Well...even smallest value of this range is much larger than the actual pixel size for a 17um pitch technology...

Again, I'm not good enough with optics, but the result looks a bit weird for me, I didn't expect such values. Any ideas what am I doing or interpreting wrong? 

One minor detail. The focal length given on a lens is a value when focusing your camera at infinity. In most cases a so called effective focal length should be used instead, also when calculating Airy disk diameter.

Max

[VGN]

One minor detail. The focal length given on a lens is a value when focusing your camera at infinity. In most cases a so called effective focal length should be used instead, also when calculating Airy disk diameter.

I'll be grateful if you explain it in more detail and attach a picture if possible. Effective Focal Length (EFL) is a bit overloaded term. The resulting focal length of multiple lenses is called EFL. Also photographers call EFL a focal length affected by the camera’s crop factor. But looks like you are talking about other things.


Assembled both boards. Everything looks to be ok. I like new buttons much more than previous ones. Magnet ring fits pefrectly, though without magnets, as I haven't received them yet.

[Vipitis]

The buttons remind me of the Blackmagic Design Micro Cinema Camera.

[Max Planck]

One minor detail. The focal length given on a lens is a value when focusing your camera at infinity. In most cases a so called effective focal length should be used instead, also when calculating Airy disk diameter.

I'll be grateful if you explain it in more detail and attach a picture if possible. Effective Focal Length (EFL) is a bit overloaded term. The resulting focal length of multiple lenses is called EFL. Also photographers call EFL a focal length affected by the camera’s crop factor. But looks like you are talking about other things.

Please look at the link below.

https://www.optowiki.info/glossary/working-f-number/

 The article is about the effective/working f-number, but basically it is about the same problem, i.e. not focusing your lens at infinity.

If you are calculating the Airy disk diameter as 2.44 x wavelength of light x f-number, you should use the working f-number, not the one given on the lens.

One more thing, stemmer imaging published several tutorials about choosing the correct lens.


Max

[Bill W]

Fortunately only a big issue in extreme close up work with long lenses.

An 8mm lens focussed down to 100mm is only reduced in illumination / field of view by 10% as the image is spread out

Bill

[VGN]

Finally received all components and PCBs!

[rockwell]

Can't wait to see the boards populated and heading to first tests.

[VGN]

Can't wait to see the boards populated and heading to first tests.

Me too)


I started from the M-board. As I have made too many changes to the power system since previous hardware revision, I decided to check power supply first, before assembling RAM and FPGA. Generally, new power system works fine!